← Xvisio SDK Depth
← Xvisio Features and Quick Start
← Xvisio SDK Documentation Home Page
Xvisio SeerSense™ DS80 series products support SGBM（passive binocular vision） depth algorithm. Refer to the corresponding datasheet of other products to confirm whether it supports SGBM depth algorithm.
Support two resolutions:
The depth image quality of 720P resolution will be higher than VGA resolution. Please refer to the product datasheet or ask Xvisio FAE personnel for the differences of the image quality.
Resolution definition:
enum class Resolution {
SGBM_640x480 = 0, ///< SGBM 480p
SGBM_1280x720 = 1, ///< SGBM 720p
};
Set API example:
device->sgbmCamera()->setSgbmResolution(xv::SgbmCamera::Resolution::SGBM_640x480);
Xvisio SDK provides some parameter configuration interfaces and configuration structure definitions:
/**
* @brief SGBM CONFIG STRUCT
*/
struct sgbm_config
{
int32_t enable_dewarp;
float dewarp_zoom_factor;
int32_t enable_disparity;
int32_t enable_depth;
int32_t enable_point_cloud;
float baseline;
float fov;
uint8_t disparity_confidence_threshold;
float homography[9];
int32_t enable_gamma;
float gamma_value;
int32_t enable_gaussian;
uint8_t mode;//standard 0 /Default:LRcheck 1 /Extended 2 /Subpixel 3
uint16_t max_distance;//mm
uint16_t min_distance;//mm
};
Note:
Set API examples:
xv::sgbm_config global_config = {...};
device->sgbmCamera()->start(global_config);
Or pass an empty string directly to use the default configuration:
device->sgbmCamera()->start("");
The frame rate and resolution of SGBM depth can't be set.
| Data Type | Resolution | Frame rate（fps） | USB bandwidth（max） |
|---|---|---|---|
| Depth Data | VGA | 60 | 281.25Mbps |
| Depth Data | 720P | 30 | 421.875Mbps |
| Point Cloud Data | VGA | 60 | 843.75Mbps |
| Point Cloud Data | 720P | 30 | 1265.25Mbps |
Note: point cloud data can not only enable the output of the device's point cloud data through USB, but also call the SDK API for deep point cloud conversion.
The image source of SGBM is binocular fisheye camera, actually it is a shared image source with SLAM. So the exposure strategy will also affect each other. Currently, it is default to enable the automatic exposure strategy in the device, and the default maximum exposure time is 3ms.
When SGBM and SLAM work simultaneously or only SGBM works, the maximum exposure time will be switched to 8ms. When only SLAM works, the maximum exposure time will be maintained at 3ms.
In addition, Xvisio SDK provides setting interface for the maximum exposure time. API example:
/*
* @brief Exposure setting.
* @param[in] aecMode 0:auto exposure 1:manual exposure
* @param[in] exposureGain Only valid in manual exposure mode, [0,255]
* @param[in] exposureTimeMs Only valid in manual exposure mode, in milliseconds
bool setExposure( int aecMode = 0, int exposureGain = 0, float exposureTime = 0.0 )
*/
//Examples:
device->fisheyeCameras()->setExposure(0); //auto exposure
//Or:
device->fisheyeCameras()->setExposure(1,20,5); //manual exposure
SGBM depth data and point cloud data can be obtained through the registration callback. The data format is 16 bit depth data (unit:mm).
/**
* @brief SGBM data.
* @note Length, width and depth are in millimeters.
*/
struct SgbmImage {
enum class Type { Disparity = 0, Depth, PointCloud};
const Type type;
explicit SgbmImage(Type t) : type(t) {}
std::size_t width = 0; //!< width of the image (in pixel)
std::size_t height = 0; //!< height of the image (in pixel)
std::shared_ptr<const std::uint8_t> data = nullptr; //! data of SGBM
unsigned int dataSize = 0;
double hostTimestamp = std::numeric_limits<double>::infinity(); //!< host timestamp of the physical measurement (in second based on the `std::chrono::steady_clock`).
std::int64_t edgeTimestampUs = (std::numeric_limits<std::int64_t>::min)(); //!< timestamp of the physical measurement (in microsecond based on edge clock).
/**
* @brief Convert to a #xv::RgbImage
*/
RgbImage toRgb() const;
};
Note:
Get SgbmImage by register callback way. The interface call process is as follows:
Xvisio SDK provides API which can convert depth data to point cloud data format.
/**
* @brief A point cloud of 3D points.
*/
struct PointCloud {
double hostTimestamp = std::numeric_limits<double>::infinity(); //!< host timestamp of ? (in second based on the `std::chrono::steady_clock`).
std::int64_t edgeTimestampUs = (std::numeric_limits<std::int64_t>::min)(); //!< timestamp of ? (in microsecond based on edge clock).
std::vector<Vector3f> points;
};
Notes:
To get Pointcloud, firstly you should obtain the SgbmImage, and then call the interface for conversion. The process is as follows:
device->tofCamera()->depthImageToPointCloud(sgbm_image)std::shared_ptr<xv::Device> device = nullptr;auto device_list = xv::getDevices(10., "");if (device_list.empty())`
{
LOG(LOG_Function,"initDevice faiiled:Timeout for device detection.");
return false;
}
device = device_list.begin()->second;device->sgbmCamera()->setSgbmResolution(xv::SgbmCamera::Resolution::SGBM_640x480);
struct xv::sgbm_config global_config = {
1 ,//enable_dewarp
1.0, //dewarp_zoom_factor
0, //enable_disparity
1, //enable_depth
0, //enable_point_cloud
0.08, //baseline
96, //fov
255, //disparity_confidence_threshold
{1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0}, //homography
1, //enable_gamma
2.2, //gamma_value
0, //enable_gaussian
0, //mode
8000, //max_distance
100, //min_distance
};
int sgbmId = -1;
sgbmId = device->sgbmCamera()->registerCallback([&](xv::SgbmImage const & sgbmDepthImage){
if(sgbmDepthImage.type == xv::SgbmImage::Type::Depth)
{
//Operation on sgbmDepthImage
//...
//point cloud process
auto points = device->sgbmCamera()->depthImageToPointCloud(sgbmDepthImage)->points;
for (auto iter = points.begin(); iter != points.end();iter++)
{
auto point = *iter;
printf(buff, "x=%f ,y=%f ,z=%f\n", point[0], point[1], point[2]);
//...
}
}
else if(sgbmDepthImage.type == SgbmImage::Type::PointCloud )//The enable_point_cloud item in sgbm_config must be enabled to enable PointCloud reporting.
{
//Operation on sgbm PointCloud
//...
}
});
//start tof,If you use the default config can call: device->sgbmCamera()->start("");
device->sgbmCamera()->start(global_config);
device->sgbmCamera()->unregisterCallback(sgbmId);
device->sgbmCamera()->stop();
← Xvisio SDK Depth
← Xvisio Features and Quick Start
← Xvisio SDK Documentation Home Page