Camera driver&V4L2 Driver Architecture

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

Based on Mavell Pxa920

1.Camera introduction

1.1. mobile phone Camera

Mobile phone digital camera function refers to the mobile phone to shoot static pictures or videos with built-in digital camera, as a new additional feature phone, mobile phone's camera has been rapid development.

Mobile phone camera is divided into internal and external, built-in camera refers to the camera in the mobile phone, more convenient. External mobile phone through the data line or mobile phone and digital camera connected to the lower interface, to complete the digital camera shooting function of all.

The performance at the stage of development of the mobile phone digital camera should be in the primary stage, with optical zoom mobile phone domestic sales do not have this function, but with the development of mobile phone digital camera, mobile phone with optical zoom will gradually appear on the market, but most of them have a digital zoom function.

At present, mobile phone digital camera function mainly includes the capture still images, video shooting, shot, shot can rotate, auto white balance, built-in flash and so on. Mobile phone camera function and material of screen, screen resolution, pixel camera, camera quality is directly related to.

1.2. Camera technical indicators

1.2.1.    JPEG image compression method

(joint photographic expert group)Static image compression method. A lossy image compression. The greater the compression ratio, image quality is poor. When the image accuracy is not high storage space is limited, can choose this format. At present, most digital cameras use the JPEG format.

1.2.2. image noise

Refers to the image noise interference, as a fixed color noise in images.

1.2.3.    From the perspective of

Eye imaging and person is the similarity principle, is simply the imaging range.

1.2.4. white balance processing technology(AWB)

Requirements in different temperature environment, as white objects, image of the screen should also is white. The color representation of spectral composition, the color of the light. Color temperature low said long wave components. When the temperature changes, the three primary colors (red, green, light blue) ratio will change, need to adjust the proportion of three primary colors to achieve color balance, this is the real white balance adjustment.

Image data image sensor is read, operation correction system will edge distortion for lens on it, and then through the bad image processing was sent in to white balance (in different lighting conditions, the human eye can put some "white" color of the objects are as white, is because the human eye is modified. But SENSOR has no such function, therefore the signal output by SENSOR need to be modified, this is the white balance processing technology).

1.2.5. power supply

The camera internal power good is to ensure a stable factors camera work.

1.2.6. color depth (color figure)

Reflect on the color recognition and image color expression ability, is to use many bits of the binary numbers to record three of the primary colors. Actually A/D converter quantization precision, refers to the signal is divided into many levels, color digit (bit) representation. The higher color depth, image color obtained is more beautiful and moving. Non professional SENSOR is generally 24; professional SENSOR is at least 36. The 24 bit SENSOR, the brightness of light sensitive unit can record the value up to 2^8=256 level, each kind of color with a 8 bit binary numbers to record, recorded the largest color is 256× 256 256× 16, 770000.

The 36 bit SENSOR, the brightness of light sensitive unit can record the value up to 2^12=4096 level, each kind of color with a 12 bit binary numbers to record, recorded the largest color is 4096× 4096 4096× 6870000000.

1.2.7. input / output interface(IO)

The serial interface (RS232/422): the slow transfer rate, 115kbit/s.

Parallel interface (PP): rate can reach 1Mbit/s.

Infrared interface (IrDA): rate is 115kbit/s, the general notebook computer with this interface.

Universal serial bus USB: plug and play interface, support hot plug. USB1.1 12Mbit/s USB2.0 rate, up to 480bit/s.

IEEE1394 (FireWire) interface (also known as ilink): the transfer rate up to 100M~400Mbit/s.


The 1.2.8. image format(image Format/ Color space)

RGB24,I420 is the most commonly used two kinds of image format.

RGB24: R, G, B three kinds of color 8bit, maximum performance of color.

I420: One of the YUV format.

Other formats: RGB565, RGB444, YUV4:2:2 etc.

1.2.9. resolution(Resolution)

The so-called resolution refers to the resolution of the numerical picture, composed of many pixels is greater, the image is more clear. Not only the display size and resolution, but also by the tube pitch, the video bandwidth and other factors. We usually see a resolution performance in the multiplication form, such as 1024*768, one of the 1024 display screen horizontal points, 768 vertical points.

QXGA (2048 X 1536)Also called 3000000 pixels

UXGA (1600X 1200)Also called 2000000 pixels

SXGA(1280 x1024)Also called 1300000 pixels

XGA(1024 x768)Also called 800000 pixels

SVGA(800 x600)Also called 500000 pixels

VGA(640x480)Also called 300000 pixels (350000 refers to the 648X488)

CIF(352x288) Also called 100000 pixels




Note that 1.3

1.3.1. Mipi interface description:

MIPI=Mobile Industry Processor Interface, Is a kind of interface of a LVDS is similar to SMIA, mainly used in mobile phone Camera Module.

As in the case of CameraModule, now Micorn and OV were introduced to support the MIPI interface of the Sensor such as Micorn MT9D112, MT9T111 and OV OV2650, for the low pixel Sensor appears to be MIPI's advantage is not very obvious, but in more than 3MP may have the advantage.

2.Linux Camera Driver

The 2.1. code structure

Ov5640 as an example, other carmera related carmere driver similarly.


2.2. interface to achieve that

    ccic_set_clock_mipi();  //CCIC Mipi clock settings
    pdata->power_on(1, 1);  //power on camera
    i2c_set_clientdata(client, info); //Initialize the I2c resource
    ov5640_mipi_detect(client);  //Camera detection
    ccic_sensor_attach(client);  //CCIC host end of the sensor driver binding

Description: the initialization function calls the camera driver, including camera power, CCIC controller MPI clock settings, I2C port initialization, camera detection and binding.

    v4l2_chip_ident_i2c_client () 
    ov5640_mipi_enum_fmsize () 
    ov5640_mipi_querycap ()   
    ov5640_mipi_enum_fmt () 
    ov5640_mipi_try_fmt ()

Description: the camera iocontorl function

v4l2_chip_ident_i2c_client() //Assign an independent field camera, because the distinction with dual camera

ov5640_mipi_reset //Camera restart function

ov5640_mipi_enum_fmsize // Image size enumeration of camera support priver and capture

ov5640_mipi_querycap //Key string assignment corresponds to the Hal layer recognition in camera device

ov5640_mipi_enum_fmt //Image format support enumeration camera

ov5640_mipi_try_fmt () //According to the image format and size of the incoming image is calculated, the width and height of the bytes consumed

ov5640_mipi_s_fmt() //According to the image format and size of the incoming, corresponding to the camera register set

ov5640_mipi_s_ctrl() //Reservation function, set some specific camera, corresponding register set, now part of this function has been implemented in Hal layer.

ov5640_mipi_g_ctrl()//Reservation function, obtain the camera settings corresponding to some effects of the register settings, now part of this function has been implemented in Hal layer.

ov5640_mipi_s_input() //Camera register initialization

ov5640_mipi_streamon() //The camera data stream

ov5640_mipi_streamoff() //The camera data

ov5640_mipi_g_register() //Read the camera of a register value

ov5640_mipi_s_register() //Set the camera of a register value

The 3 Host controller (based on V4L2)

3.1. V4L2 knowledge

Video4linux2 (V4L2), is a video device driver in linux. In Linux, the video device is the device file, can be accessed like ordinary files to read and write to the camera, in /dev/video0

3.1.1. process (video):

1 open the device file. int fd=open("/dev/video0″,O_RDWR);

2 equipment capability, have a look what equipment has functions, such as whether a video input, or audio input and output. VIDIOC_QUERYCAP,struct v4l2_capability

System and the frame format of 3 set video formats including PAL, NTSC, frame format, including the width and height.

VIDIOC_S_STD,VIDIOC_S_FMT,struct v4l2_std_id,struct v4l2_format

4 to drive the application frame buffer, usually not more than 5. struct v4l2_requestbuffers

5 will apply to the frame buffer is mapped to user space, so it can be operated directly collected frame, without having to copy. mmap

6 the frame buffer to apply to all incoming queue, to store the collected data in.VIDIOC_QBUF,struct v4l2_buffer

7 Video Collection. VIDIOC_STREAMON

8 out of the queue to get collected frame buffer data, made the original data collection. VIDIOC_DQBUF

9 the buffer re entry queue tail, so you can cycle sampling. VIDIOC_QBUF

The 10 stop video collection. VIDIOC_STREAMOFF

11 close video equipment. close(fd);

3.1.2. in V4L2 development, the general will use the following command identifier:

1 VIDIOC_REQBUFS: Allocate memory

2 VIDIOC_QUERYBUF: The distribution of VIDIOC_REQBUFS in the data cache into a physical address

3 VIDIOC_QUERYCAP: Query driven function

4 VIDIOC_ENUM_FMT: Gets the current drive supported video formats

5 VIDIOC_S_FMT: Set the current driving frequency capture format

6 VIDIOC_G_FMT: Read the current driving frequency capture format

7 VIDIOC_TRY_FMT: Verify that the current driven display format

8 VIDIOC_CROPCAP: Query driven pruning ability

9 VIDIOC_S_CROP: Frame of video signal

10 VIDIOC_G_CROP: Read the video signal

11 VIDIOC_QBUF: The data read from the cache

12 VIDIOC_DQBUF: The data back into the buffer queue

13 VIDIOC_STREAMON: Start a video display function

14 VIDIOC_STREAMOFF: The end of the video display function

15 VIDIOC_QUERYSTD: Check the current video equipment in support of standards, such as PAL or NTSC.

These IO calls, some is necessary, some are optional.

3.2. Host controller (video device) code structure

Here mainly introduce 920 host end of the camera video controller, which is based on the v4l2 architecture, code file, see pxa910_camera.c.


An important interface 3.3. realization

If the structure shown in Figure 3.2, the camera host is regarded as the standard v4l2 framework to register, that is in accordance with the struct video_device structure (standard video settings) to register the device to the system.

    platform_set_drvdata(pdev, cam);
    request_irq()  // CCIC IRQ interrupt register
    ccic_ctlr_init()    //CCIC controller initialization

Description: the initialization function call to start the camera video devices

platform_set_drvdata(pdev, cam) : //Connect the device object and device driver

init_waitqueue_head()  : //Declare wait events, see the ccic_v4l_poll function processing

request_irq(): CCIC IRQ interrupt register, the corresponding interrupt processing function ccic_irq () function

ccic_ctlr_init() : CCIC controller initialization

video_register_device : //V4l2 video device registration


v4l2 videoMainly to achieve ccic_v4l_fops : (struct v4l2_file_operations) Andccic_ioctl_ops : (struct v4l2_ioctl_ops)

static const struct v4l2_file_operations ccic_v4l_fops = {
    .owner = THIS_MODULE,
    .open = ccic_v4l_open,  //open files
    .release = ccic_v4l_release, //release files resource
    .poll = ccic_v4l_poll,  //The files poll function for the interface
    .mmap = ccic_v4l_mmap, //Files memory mapped interface
    .ioctl = ccic_v4l_ioctl, //Files corresponds to the iocontorl operation interface

Explain: v4l2 videoAs the file related operations interface corresponding

struct v4l2_ioctl_ops ccic_ioctl_ops = {
    .vidioc_querycap        = ccic_vidioc_querycap,
    .vidioc_enum_fmt_vid_cap= ccic_vidioc_enum_fmt_cap,
    .vidioc_try_fmt_vid_cap = ccic_vidioc_try_fmt_cap,
    .vidioc_s_fmt_vid_cap   = ccic_vidioc_s_fmt_cap,
    .vidioc_g_fmt_vid_cap   = ccic_vidioc_g_fmt_cap,
    .vidioc_enum_input      = ccic_vidioc_enum_input,
    .vidioc_g_input         = ccic_vidioc_g_input,
    .vidioc_s_input         = ccic_vidioc_s_input,
    .vidioc_s_std           = ccic_vidioc_s_std,
    .vidioc_reqbufs         = ccic_vidioc_reqbufs,
    .vidioc_querybuf        = ccic_vidioc_querybuf,
    .vidioc_qbuf            = ccic_vidioc_qbuf,
    .vidioc_dqbuf           = ccic_vidioc_dqbuf,
    .vidioc_streamon        = ccic_vidioc_streamon,
    .vidioc_streamoff       = ccic_vidioc_streamoff,
    .vidioc_queryctrl       = ccic_vidioc_queryctrl,
    .vidioc_g_ctrl          = ccic_vidioc_g_ctrl,
    .vidioc_s_ctrl          = ccic_vidioc_s_ctrl,
    .vidioc_g_parm          = ccic_vidioc_g_parm,
    .vidioc_s_parm          = ccic_vidioc_s_parm,
    .vidioc_cropcap         = ccic_vidioc_cropcap,
    .vidioc_g_register      = ccic_vidioc_g_register,
    .vidioc_s_register      = ccic_vidioc_s_register,


Description: processing function v4l2 video equipment IO contorl correspondence.

ccic_vidioc_querycap() //Query driven function, get the camera property

ccic_vidioc_enum_fmt_cap() Gets the current drive supported video formats

ccic_vidioc_try_fmt_cap() //The corresponding call ov5640_mipi_try_fmt () function interface, see Chapter

ccic_vidioc_s_fmt_cap() //Set the current driving frequency capture format

ccic_vidioc_g_fmt_cap() //Read the current driving frequency capture format

ccic_vidioc_enum_input() //Detection of camera support, fault tolerance

ccic_vidioc_g_input() //Gets a camera ID (300000 sensor_selected 0, 5000000 sensor_selected 1)

ccic_vidioc_s_input() //To set the camera ID switch

ccic_vidioc_reqbufs() //Allocate buf memory

ccic_vidioc_querybuf, //The distribution of ccic_vidioc_reqbufs in the data cache into a physical address

ccic_vidioc_qbuf() //The buffer to buffer queue

ccic_vidioc_dqbuf() //The data read from the cache

ccic_vidioc_streamon() //Start a video display function

ccic_vidioc_streamoff() // The end of the video display function

ccic_vidioc_queryctrl() //The corresponding call ov5640_mipi_queryctrl () function interface, see Chapter

ccic_vidioc_g_ctrl() //Calls the corresponding ov5640_mipi_ g_ctrl () function interface, see Chapter

ccic_vidioc_s_ctrl() //Calls the corresponding ov5640_mipi_ s_ctrl () function interface, see Chapter

ccic_vidioc_cropcap() //Reservation interface, query driven pruning ability

ccic_vidioc_g_register() //The corresponding call ov5640_mipi_g_registe () function interface, see Chapter

ccic_vidioc_s_register() //Calls the corresponding ov5640_mipi_ s_register () function interface, see Chapter

Note that 3.4

Camera video data flow has the following main correlation function:

ccic_vidioc_reqbufs() //Allocate memory

ccic_vidioc_querybuf, //The distribution of ccic_vidioc_reqbufs in the data cache into a physical address

ccic_vidioc_qbuf() //The buffer to buffer queue

ccic_vidioc_dqbuf() //The data read from the cache

ccic_irq() //CCIC interrupt function

ccic_v4l_poll, //The files poll function for the interface

check_jpeg_header //Grab the data detection is an JPEG file

Specific how data flows, see below: (from the marrvel CCIC sensor bring up PXA920 guide.pdf)

Description: the VIDIOC_QBUF function ccic_vidioc_qbuf()

      The VIDIOC_QBUF function ccic_vidioc_dqbuf()

      Dma_map_page //The DMA buffer data to the kernel memory space

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download

Posted by Harlan at February 20, 2014 - 12:03 AM