如何給點雲紋理?

[英]How to give texture to a point cloud?


I have a disparity image obtained using OpenCV. I am able to display a 640 x 360 grid of points using OpenGL and CUDA, and give to each point a Z value which corresponds to the disparity value. The result:

我有一個使用OpenCV獲得的視差圖像。我能夠使用OpenGL和CUDA顯示640 x 360網格點,並為每個點提供與視差值對應的Z值。結果:

screenshot

Now I wish to give to each point in the point cloud a colour value corresponding to the pixel in the left image of the stereo camera which have same size (640 x 380).

現在我希望給點雲中的每個點賦予與立體相機的左圖像中具有相同尺寸(640×380)的像素相對應的顏色值。

This is my CUDA kernel and the calling function where I have both images, the disparity and the left image (gray scale) containing the color information:

這是我的CUDA內核和調用函數,其中我有兩個圖像,差異和包含顏色信息的左圖像(灰度):

__global__ void simple_vbo_kernel(float4 *pos, unsigned int width, unsigned int height, float time,
        uchar* disp, int stepDisp)
{
    const int x = threadIdx.x + blockIdx.x * blockDim.x;
    const int y = threadIdx.y + blockIdx.y * blockDim.y;
     if ((x < width) && (y < height))
     {

            float u = x / (float) width;
            float v = y / (float) height;
            u = u*2.0f - 1.0f;
            v = v*2.0f - 1.0f;

            const int depth  = y * stepDisp + x;
            float w =  static_cast<float>(disp[depth]);
            w/=10;

            pos[y*width+x] = make_float4(u, w, v, 1.0f);
     }

}

extern "C"
void launch_kernel(dim3 grid, dim3 block, float4 *pos, unsigned int mesh_width,unsigned int mesh_height,
        float time, cv::Mat disp, cv::Mat left)
{
    if(!disp.empty() && !left.empty()){

        uchar* d_image;
        int dsize = disp.rows * disp.step;
        cudaMalloc((void**)&d_image, dsize);
        cudaMemcpy(d_image, disp.ptr(), dsize, cudaMemcpyHostToDevice);

        simple_vbo_kernel<<< grid, block>>>(pos, mesh_width, mesh_height, time, d_image, disp.step);
        cudaDeviceSynchronize();
        cudaFree(d_image);
    }
}

My question is, what is the easiest way to give a texture or colour to the points in the point cloud, in this case using the cv::Mat left? I have seen some other examples from the CUDA samples but I did not find how to do it.

我的問題是,給點雲中的點提供紋理或顏色的最簡單方法是什么,在這種情況下使用cv :: Mat左邊?我已經看到了CUDA樣本中的其他一些例子,但我沒有找到如何做到這一點。

2 个解决方案

#1


In case of sparse point clouds (like your's) usually the best course of action is adding a colour attribute to the vertices and assign it the desired color. Make the drawing shaders pass the value of that attribute to the emited fragment color.

在稀疏點雲(如您的)的情況下,通常最好的做法是向頂點添加顏色屬性並為其指定所需的顏色。使繪圖着色器將該屬性的值傳遞給已發出的片段顏色。

#2


I found a solution. In the kernel part I have a new uchar4 mapped input for the color, both input pointers must have same size for a correct relation between point and color (float4 *pos and uchar4 *color). Since d_image is a gray scale image (disparity image) and the one to give texture is a color image, this should be taken into account for the way to access to each array. Opencv it is in BGR order and gstreamer in RGB:

我找到了解決方案。在內核部分,我有一個新的uchar4映射輸入顏色,兩個輸入指針必須具有相同的大小,以便在點和顏色之間建立正確的關系(float4 * pos和uchar4 * color)。由於d_image是灰度圖像(視差圖像),而賦予紋理的圖像是彩色圖像,因此應該考慮到訪問每個數組的方式。 Opencv它是BGR順序和RGB中的gstreamer:

__global__ void simple_vbo_kernel(float4 *pos, uchar4 *color, unsigned int width, unsigned int height, uchar* d_image, int stepDisp, uchar* d_color, int stepLeft)
{
    const int x = threadIdx.x + blockIdx.x * blockDim.x;
    const int y = threadIdx.y + blockIdx.y * blockDim.y;
     if ((x < width) && (y < height)){

            float u = x / (float) width;
            float v = y / (float) height;
            u = u*2.0f - 1.0f;
            v = v*2.0f - 1.0f;

            const int depthi  = y * stepDisp + x;
            float w =  static_cast<float>(d_image[depthi]);
            w/=25;
            pos[y*width+x] = make_float4(u, w, v, 1.0f);

            uchar b =  static_cast<uchar>(d_color[y*width*3+x*3 + 0]);
            uchar g =  static_cast<uchar>(d_color[y*width*3+x*3 + 1]);
            uchar r =  static_cast<uchar>(d_color[y*width*3+x*3 + 2]);
            color[y*width+x] = make_uchar4( r, g, b, 1.0f);
     }

}

extern "C"
void launch_kernel(dim3 grid, dim3 block, unsigned int mesh_width, unsigned int mesh_height, float4 *pos,  cv::Mat disp, uchar4 *color,  cv::Mat left)
{
    if(!disp.empty() && !left.empty()){

        uchar* d_image;
        int dsize = disp.rows * disp.step;
        cudaMalloc((void**)&d_image, dsize);
        cudaMemcpy(d_image, disp.ptr(), dsize, cudaMemcpyHostToDevice);

        uchar* d_color;
        int dcolorsize = left.rows * left.step;
        cudaMalloc((void**)&d_color, dcolorsize);
        cudaMemcpy(d_color, left.ptr(), dcolorsize, cudaMemcpyHostToDevice);

    simple_vbo_kernel<<< grid, block>>>(pos, color, mesh_width, mesh_height, d_image, disp.step, d_color, left.step);
    cudaDeviceSynchronize();
    cudaFree(d_image);
    cudaFree(d_color);

    }
}

Calling to the kernel:

調用內核:

void runCuda(struct cudaGraphicsResource **vbo_resource, Mat dsp, struct cudaGraphicsResource **vbo_resource_color, Mat lft)
{
    float4 *dptr_pos;
    checkCudaErrors(cudaGraphicsMapResources(1, vbo_resource, 0));
    size_t num_bytes;
    checkCudaErrors(cudaGraphicsResourceGetMappedPointer((void **)&dptr_pos, &num_bytes, *vbo_resource));

    uchar4 *dptr_color;
    checkCudaErrors(cudaGraphicsMapResources(1, vbo_resource_color, 0));
    size_t num_bytes_color;
    checkCudaErrors(cudaGraphicsResourceGetMappedPointer((void **)&dptr_color, &num_bytes_color, *vbo_resource_color));

    dim3 block(8, 8, 1);
    dim3 grid(mesh_width / block.x, mesh_height / block.y, 1);
    launch_kernel(grid,block, mesh_width, mesh_height, dptr_pos, dsp, dptr_color, lft);

    checkCudaErrors(cudaGraphicsUnmapResources(1, vbo_resource, 0));
    checkCudaErrors(cudaGraphicsUnmapResources(1, vbo_resource_color, 0));
}

Also important in the initialization of the OpenGl add this:

在OpenGl的初始化中同樣重要的是添加:

glEnable( GL_TEXTURE_2D );

and also enable depth test. I had Disabled before and I have strange image overlapings:

並且還可以進行深度測試。我之前有殘疾,我有奇怪的圖像重疊:

glEnable(GL_DEPTH_TEST);

And the part where the image must be shown or updated, in my case I have it in my glutDisplayFunc(display) function:

並且必須顯示或更新圖像的部分,在我的情況下,我在glutDisplayFunc(顯示)函數中有它:

    glBindBuffer(GL_ARRAY_BUFFER, pos_vbo);
    glVertexPointer(4, GL_FLOAT, 0, 0);
    glEnableClientState(GL_VERTEX_ARRAY);

    glBindBuffer(GL_ARRAY_BUFFER, color_vbo);
    glColorPointer(4, GL_UNSIGNED_BYTE, 0, 0);
    glEnableClientState(GL_COLOR_ARRAY);

    glPointSize(3.0);
    glDrawArrays(GL_POINTS, 0, mesh_width * mesh_height);

    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_COLOR_ARRAY);
    glBindBuffer(GL_ARRAY_BUFFER, 0);

    glutSwapBuffers();
    glFlush();
    glDeleteTextures(1, &color_vbo);

My image is a grayscale image because I am still trying to capture my rtsp stream with gstreamer 0.10 in color http://postimg.org/image/ub8ds4zw5/

我的圖像是灰度圖像,因為我仍然試圖用gstreamer 0.10顏色捕獲我的rtsp流http://postimg.org/image/ub8ds4zw5/

I have opened a new post trying to solve the color issue:

我打開了一個試圖解決顏色問題的新帖子:

GstBuffer to color Mat

GstBuffer為Mat着色


注意!

本站翻译的文章,版权归属于本站,未经许可禁止转摘,转摘请注明本文地址:https://www.itdaan.com/blog/2015/05/13/729d9bbcf0b30d9e32d66143014a73b5.html



 
粤ICP备14056181号  © 2014-2020 ITdaan.com