T265深度图像输出

1. T265深度图像输出

1.1. 环境依赖

  • T265摄像头
  • python3
  • pip3
  • opencv-python
  • pyrealsense2

1.2. 安装运行环境

安装秘钥

sudo apt-key adv --keyserver keys.gnupg.net --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE

增加库

sudo add-apt-repository "deb http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo xenial main" -u

安装驱动

sudo apt-get install librealsense2-dkms
sudo apt-get install librealsense2-utils

安装调试环境

sudo apt-get install librealsense2-dev
sudo apt-get install librealsense2-dbg

安装pip

sudo apt install python3-pip

安装opencv-python

pip install opencv-python

安装pyrealsense2

pip install pyrealsense2

1.3. 获取深度图

新建t265_stereo

vim t265_stereo.py

添加下面代码

程序如下

#!/usr/bin/python
# -*- coding: utf-8 -*-
## License: Apache 2.0. See LICENSE file in root directory.
## Copyright(c) 2019 Intel Corporation. All Rights Reserved.
# Python 2/3 compatibility
from __future__ import print_function

"""
This example shows how to use T265 intrinsics and extrinsics in OpenCV to
asynchronously compute depth maps from T265 fisheye images on the host.
T265 is not a depth camera and the quality of passive-only depth options will
always be limited compared to (e.g.) the D4XX series cameras. However, T265 does
have two global shutter cameras in a stereo configuration, and in this example
we show how to set up OpenCV to undistort the images and compute stereo depth
from them.
Getting started with python3, OpenCV and T265 on Ubuntu 16.04:
First, set up the virtual enviroment:
$ apt-get install python3-venv  # install python3 built in venv support
$ python3 -m venv py3librs      # create a virtual environment in pylibrs
$ source py3librs/bin/activate  # activate the venv, do this from every terminal
$ pip install opencv-python     # install opencv 4.1 in the venv
$ pip install pyrealsense2      # install librealsense python bindings
Then, for every new terminal:
$ source py3librs/bin/activate  # Activate the virtual environment
$ python3 t265_stereo.py        # Run the example
"""

# First import the library
import pyrealsense2 as rs

# Import OpenCV and numpy
import cv2
import numpy as np
from math import tan, pi

"""
In this section, we will set up the functions that will translate the camera
intrinsics and extrinsics from librealsense into parameters that can be used
with OpenCV.
The T265 uses very wide angle lenses, so the distortion is modeled using a four
parameter distortion model known as Kanalla-Brandt. OpenCV supports this
distortion model in their "fisheye" module, more details can be found here:
https://docs.opencv.org/3.4/db/d58/group__calib3d__fisheye.html
"""

"""
Returns R, T transform from src to dst
"""
def get_extrinsics(src, dst):
    extrinsics = src.get_extrinsics_to(dst)
    R = np.reshape(extrinsics.rotation, [3,3]).T
    T = np.array(extrinsics.translation)
    return (R, T)

"""
Returns a camera matrix K from librealsense intrinsics
"""
def camera_matrix(intrinsics):
    return np.array([[intrinsics.fx,             0, intrinsics.ppx],
                     [            0, intrinsics.fy, intrinsics.ppy],
                     [            0,             0,              1]])

"""
Returns the fisheye distortion from librealsense intrinsics
"""
def fisheye_distortion(intrinsics):
    return np.array(intrinsics.coeffs[:4])

# Set up a mutex to share data between threads
from threading import Lock
frame_mutex = Lock()
frame_data = {"left"  : None,
              "right" : None,
              "timestamp_ms" : None
              }

"""
This callback is called on a separate thread, so we must use a mutex
to ensure that data is synchronized properly. We should also be
careful not to do much work on this thread to avoid data backing up in the
callback queue.
"""
def callback(frame):
    global frame_data
    if frame.is_frameset():
        frameset = frame.as_frameset()
        f1 = frameset.get_fisheye_frame(1).as_video_frame()
        f2 = frameset.get_fisheye_frame(2).as_video_frame()
        left_data = np.asanyarray(f1.get_data())
        right_data = np.asanyarray(f2.get_data())
        ts = frameset.get_timestamp()
        frame_mutex.acquire()
        frame_data["left"] = left_data
        frame_data["right"] = right_data
        frame_data["timestamp_ms"] = ts
        frame_mutex.release()

# Declare RealSense pipeline, encapsulating the actual device and sensors
pipe = rs.pipeline()

# Build config object and stream everything
cfg = rs.config()

# Start streaming with our callback
pipe.start(cfg, callback)

try:
    # Set up an OpenCV window to visualize the results
    WINDOW_TITLE = 'Realsense'
    cv2.namedWindow(WINDOW_TITLE, cv2.WINDOW_NORMAL)

    # Configure the OpenCV stereo algorithm. See
    # https://docs.opencv.org/3.4/d2/d85/classcv_1_1StereoSGBM.html for a
    # description of the parameters
    window_size = 5
    min_disp = 0
    # must be divisible by 16
    num_disp = 112 - min_disp
    max_disp = min_disp + num_disp
    stereo = cv2.StereoSGBM_create(minDisparity = min_disp,
                                   numDisparities = num_disp,
                                   blockSize = 16,
                                   P1 = 8*3*window_size**2,
                                   P2 = 32*3*window_size**2,
                                   disp12MaxDiff = 1,
                                   uniquenessRatio = 10,
                                   speckleWindowSize = 100,
                                   speckleRange = 32)

    # Retreive the stream and intrinsic properties for both cameras
    profiles = pipe.get_active_profile()
    streams = {"left"  : profiles.get_stream(rs.stream.fisheye, 1).as_video_stream_profile(),
               "right" : profiles.get_stream(rs.stream.fisheye, 2).as_video_stream_profile()}
    intrinsics = {"left"  : streams["left"].get_intrinsics(),
                  "right" : streams["right"].get_intrinsics()}

    # Print information about both cameras
    print("Left camera:",  intrinsics["left"])
    print("Right camera:", intrinsics["right"])

    # Translate the intrinsics from librealsense into OpenCV
    K_left  = camera_matrix(intrinsics["left"])
    D_left  = fisheye_distortion(intrinsics["left"])
    K_right = camera_matrix(intrinsics["right"])
    D_right = fisheye_distortion(intrinsics["right"])
    (width, height) = (intrinsics["left"].width, intrinsics["left"].height)

    # Get the relative extrinsics between the left and right camera
    (R, T) = get_extrinsics(streams["left"], streams["right"])

    # We need to determine what focal length our undistorted images should have
    # in order to set up the camera matrices for initUndistortRectifyMap.  We
    # could use stereoRectify, but here we show how to derive these projection
    # matrices from the calibration and a desired height and field of view

    # We calculate the undistorted focal length:
    #
    #         h
    # -----------------
    #  \      |      /
    #    \    | f  /
    #     \   |   /
    #      \ fov /
    #        \|/
    stereo_fov_rad = 90 * (pi/180)  # 90 degree desired fov
    stereo_height_px = 300          # 300x300 pixel stereo output
    stereo_focal_px = stereo_height_px/2 / tan(stereo_fov_rad/2)

    # We set the left rotation to identity and the right rotation
    # the rotation between the cameras
    R_left = np.eye(3)
    R_right = R

    # The stereo algorithm needs max_disp extra pixels in order to produce valid
    # disparity on the desired output region. This changes the width, but the
    # center of projection should be on the center of the cropped image
    stereo_width_px = stereo_height_px + max_disp
    stereo_size = (stereo_width_px, stereo_height_px)
    stereo_cx = (stereo_height_px - 1)/2 + max_disp
    stereo_cy = (stereo_height_px - 1)/2

    # Construct the left and right projection matrices, the only difference is
    # that the right projection matrix should have a shift along the x axis of
    # baseline*focal_length
    P_left = np.array([[stereo_focal_px, 0, stereo_cx, 0],
                       [0, stereo_focal_px, stereo_cy, 0],
                       [0,               0,         1, 0]])
    P_right = P_left.copy()
    P_right[0][3] = T[0]*stereo_focal_px

    # Construct Q for use with cv2.reprojectImageTo3D. Subtract max_disp from x
    # since we will crop the disparity later
    Q = np.array([[1, 0,       0, -(stereo_cx - max_disp)],
                  [0, 1,       0, -stereo_cy],
                  [0, 0,       0, stereo_focal_px],
                  [0, 0, -1/T[0], 0]])

    # Create an undistortion map for the left and right camera which applies the
    # rectification and undoes the camera distortion. This only has to be done
    # once
    m1type = cv2.CV_32FC1
    (lm1, lm2) = cv2.fisheye.initUndistortRectifyMap(K_left, D_left, R_left, P_left, stereo_size, m1type)
    (rm1, rm2) = cv2.fisheye.initUndistortRectifyMap(K_right, D_right, R_right, P_right, stereo_size, m1type)
    undistort_rectify = {"left"  : (lm1, lm2),
                         "right" : (rm1, rm2)}

    mode = "stack"
    while True:
        # Check if the camera has acquired any frames
        frame_mutex.acquire()
        valid = frame_data["timestamp_ms"] is not None
        frame_mutex.release()

        # If frames are ready to process
        if valid:
            # Hold the mutex only long enough to copy the stereo frames
            frame_mutex.acquire()
            frame_copy = {"left"  : frame_data["left"].copy(),
                          "right" : frame_data["right"].copy()}
            frame_mutex.release()

            # Undistort and crop the center of the frames
            center_undistorted = {"left" : cv2.remap(src = frame_copy["left"],
                                          map1 = undistort_rectify["left"][0],
                                          map2 = undistort_rectify["left"][1],
                                          interpolation = cv2.INTER_LINEAR),
                                  "right" : cv2.remap(src = frame_copy["right"],
                                          map1 = undistort_rectify["right"][0],
                                          map2 = undistort_rectify["right"][1],
                                          interpolation = cv2.INTER_LINEAR)}

            # compute the disparity on the center of the frames and convert it to a pixel disparity (divide by DISP_SCALE=16)
            disparity = stereo.compute(center_undistorted["left"], center_undistorted["right"]).astype(np.float32) / 16.0

            # re-crop just the valid part of the disparity
            disparity = disparity[:,max_disp:]

            # convert disparity to 0-255 and color it
            disp_vis = 255*(disparity - min_disp)/ num_disp
            disp_color = cv2.applyColorMap(cv2.convertScaleAbs(disp_vis,1), cv2.COLORMAP_JET)
            color_image = cv2.cvtColor(center_undistorted["left"][:,max_disp:], cv2.COLOR_GRAY2RGB)

            if mode == "stack":
                cv2.imshow(WINDOW_TITLE, np.hstack((color_image, disp_color)))
            if mode == "overlay":
                ind = disparity >= min_disp
                color_image[ind, 0] = disp_color[ind, 0]
                color_image[ind, 1] = disp_color[ind, 1]
                color_image[ind, 2] = disp_color[ind, 2]
                cv2.imshow(WINDOW_TITLE, color_image)
        key = cv2.waitKey(1)
        if key == ord('s'): mode = "stack"
        if key == ord('o'): mode = "overlay"
        if key == ord('q') or cv2.getWindowProperty(WINDOW_TITLE, cv2.WND_PROP_VISIBLE) < 1:
            break
finally:
    pipe.stop()

运行t265_stereo.py

python3 t265_stereo.py

原文地址:https://www.cnblogs.com/hellocxz/p/12104279.html

时间: 2024-11-06 09:58:51

T265深度图像输出的相关文章

Kienct与Arduino学习笔记(2) 深度图像和真实世界坐标系中的深度图

转载请注明出处:http://blog.csdn.net/lxk7280 首先,要接触一下KinectOrbit这个摄像机库,这篇文章中有这个库的下载网址和简单的介绍:http://blog.csdn.net/lxk7280/article/details/38184355.将下载得到的文件放到相应的Processing的一个子文件夹后,即可使用.    KinectOrbit库下的鼠标和键盘的操作方法: 1.右键拖拽:摄像机摇拍. 2.左键拖拽:绕对象旋转. 3.滚动:缩放操作. 4.P键保存

Kinect深度图像滤波

最近在做机器视觉方面的一点工作,用Kinect作sensor获取深度数据.颜色.手势识别等.非常感激CNBlog上的两篇博文:(1)独钓寒江的http://www.cnblogs.com/yangecnu/archive/2012/03/30/KinectSDK_Geting_Started.html 从中学到了不少关于在WPF平台上使用C#,利用微软Kinect SDK开发自己的应用程序的知识,给了我很大的帮助,在此表示感谢! 但是博主独钓寒江的博文中,针对深度图像滤波,只简要说了取反和用Bg

kinect2.0 基础篇第3篇 用C#在Visual Studio上编写把深度图像转换成彩色图像

本示例实现的功能有:有两个Radiobutton控件  选一个,点击启动按钮, 第一个是将深度图像转换成彩色图像 第二个是将深度图像做一些简单处理(例如太暗的调白一点) 涉及到一点遥感图像处理知识,将深度数据值转换为色调和饱和度 遥感图像处理那块不懂,有兴趣的自己可以研究研究,代码的基于kinect1的教程,慢慢尝试出来的,虽然功能实现了,但是原理还不是很懂 <Window x:Class="EnhancedDepthWPF.MainWindow" xmlns="htt

深度图像配准(Registration)原理

机器视觉中,3D相机产生的深度图像(depth image)通常需要配准(registration),生成配准深度图像(registed depth image).实际上配准的目的就是想让深度图和彩色图重合在一起,即是将深度图像的图像坐标系转换到彩色图像的图像坐标系下.下面我们来介绍其推导的过程. 为了描述方便,首先做些简单的假设.3D相机的左侧相机(left camera)为红外相机(即深度相机,ir camera),右侧相机(right camera)为彩色相机(color camera).

Matlab下imwrite,Uint16的深度图像

Matlab下imwrite,Uint16的深度图像 1. 在Matlab命令窗口输入命令: help imwrite 会有如下解释: If the input array is of class uint16 and the format supports 16-bit data (JPEG, PNG, and TIFF), imwrite outputs the data as 16-bit values. If the format does not support 16-bit valu

深度图像的获取原理

RGB-D(深度图像)   深度图像 = 普通的RGB三通道彩色图像 + Depth Map   在3D计算机图形中,Depth Map(深度图)是包含与视点的场景对象的表面的距离有关的信息的图像或图像通道.其中,Depth Map 类似于灰度图像,只是它的每个像素值是传感器距离物体的实际距离.通常RGB图像和Depth图像是配准的,因而像素点之间具有一对一的对应关系.   下面可以看到两个不同的深度图,以及从中衍生的原始模型.第一个深度图显示与照相机的距离成比例的亮度.较近的表面较暗; 其他表

python二维图像输出操作大全(非常全)!

//2019.07.141.matplotlib模块输出函数图像应用时主要用的是它的ptplot模块,因此在导入使用该模块时可以直接用以下语句:import matplotlib.pyplot as plt2.matplotlib模块输出图线时它得到坐标轴数值以及标题正常默认是英文形式,而如果需要在转换成为中文形式并且需要正常输出负号则需要用以下两句python语句设置一下,这样便可以正常输出中英文两种形式标题与带负号的坐标数值:plt.rcParams["font.sans-serif&quo

libjpeg库解码jpeg图像输出数据排列问题

上一篇文章我们谈到如何修改libjpeg库来解码内存中的jpeg数据这件事情,也确实做到了这一点,然而紧随其后的就发现了一个很蛋疼的问题.因为libjpeg库输出的图像像素是以r-g-b这样的顺序排列的,而在Windows平台上要去显示内存中的图像数据通常是在内存中创建一个位图对象bitmap,然后将要显示的图像数据拷到其相应的位置中,拷贝完成后显示图像.蛋疼的地方就在于bitmap的像素是以b-g-r这样的顺序排列的,这就要求我们在拷贝数据的时候要逐个像素点进行修改,这样做浪费的时间是跟图像的

深度图像检测算法总结与对比(1)

1. R-CNN:Rich feature hierarchies for accurate object detection and semantic segmentation 技术路线:selective search + CNN + SVMs Step1:候选框提取(selective search) 训练:给定一张图片,利用seletive search方法从中提取出2000个候选框.由于候选框大小不一,考虑到后续CNN要求输入的图片大小统一,将2000个候选框全部resize到227*