|
RTAB-Map 0.23.10
Real-Time Appearance-Based Mapping
|
Represents a pinhole camera model containing intrinsic and extrinsic parameters, used for projection, rectification, and transformation. More...
#include <CameraModel.h>
Public Member Functions | |
| CameraModel () | |
| Default constructor. | |
| CameraModel (const std::string &name, const cv::Size &imageSize, const cv::Mat &K, const cv::Mat &D, const cv::Mat &R, const cv::Mat &P, const Transform &localTransform=opticalRotation()) | |
| Constructor using full camera parameters. | |
| CameraModel (double fx, double fy, double cx, double cy, const Transform &localTransform=opticalRotation(), double Tx=0.0f, const cv::Size &imageSize=cv::Size(0, 0)) | |
| Minimal constructor using intrinsic parameters. This assumes the images are already rectified. | |
| CameraModel (const std::string &name, double fx, double fy, double cx, double cy, const Transform &localTransform=opticalRotation(), double Tx=0.0f, const cv::Size &imageSize=cv::Size(0, 0)) | |
| Minimal constructor with name for saving. | |
| virtual | ~CameraModel () |
| Destructor. | |
| bool | initRectificationMap () |
| Initializes the rectification maps used to undistort and rectify images. | |
| bool | isRectificationMapInitialized () const |
| Checks if the rectification map has been initialized. | |
| bool | isValidForProjection () const |
| Checks if the model is valid for 2D->3D projection. | |
| bool | isValidForReprojection () const |
| Checks if the model is valid for 3D->2D reprojection. | |
| bool | isValidForRectification () const |
| Checks if the model has sufficient data for image rectification. | |
| void | setName (const std::string &name) |
| Sets the camera name, used to set a camera name when saving to a file. | |
| const std::string & | name () const |
| Returns the camera name. | |
| double | fx () const |
| Returns focal length in x. | |
| double | fy () const |
| Returns focal length in y. | |
| double | cx () const |
| Returns principal point x. | |
| double | cy () const |
| Returns principal point y. | |
| double | Tx () const |
| Returns the x translation (usually fx * baseline in case of stereo, otherwise would be 0). | |
| cv::Mat | K_raw () const |
| Returns the raw intrinsic matrix (before rectification). | |
| cv::Mat | D_raw () const |
| Returns the raw distortion coefficients (before rectification). | |
| cv::Mat | K () const |
| Returns the rectified camera intrinsic matrix if the projection matrix P exists, otherwise returns the raw intrinsic matrix. | |
| cv::Mat | D () const |
| Returns the rectified distortion coefficients (1x5 filled with zeros) if the projection matrix P exists, otherwise returns the raw distortion coefficients. | |
| cv::Mat | R () const |
| Returns the rectification matrix. | |
| cv::Mat | P () const |
| Returns the projection matrix. | |
| void | setLocalTransform (const Transform &transform) |
| Sets the local transform of the camera (base frame to optical frame). | |
| const Transform & | localTransform () const |
| Returns the local transform (base frame to optical frame). | |
| void | setImageSize (const cv::Size &size) |
| Sets the image size of the camera model and updates the principal point if undefined. | |
| const cv::Size & | imageSize () const |
| int | imageWidth () const |
| int | imageHeight () const |
| double | fovX () const |
| Returns the horizontal field of view (FoV) in radians. | |
| double | fovY () const |
| Returns the vertical field of view (FoV) in radians. | |
| double | horizontalFOV () const |
| Returns the horizontal field of view in degrees. | |
| double | verticalFOV () const |
| Returns the vertical field of view in degrees. | |
| bool | isFisheye () const |
| Checks if the distortion model is fisheye (6 coefficients: k1,k2,0,0,k3,k4). | |
| bool | load (const std::string &filePath, bool initRectificationMaps=true) |
| Loads the camera model parameters from a YAML calibration file. | |
| bool | load (const std::string &directory, const std::string &cameraName, bool initRectificationMaps=true) |
| Loads the camera model by constructing a file path from a directory and camera name. | |
| bool | save (const std::string &directory) const |
| Saves the camera model parameters to a YAML calibration file in ROS format. | |
| std::vector< unsigned char > | serialize () const |
| Serializes the camera model to a binary format. | |
| unsigned int | deserialize (const std::vector< unsigned char > &data) |
| Deserializes a camera model from a byte vector. | |
| unsigned int | deserialize (const unsigned char *data, unsigned int dataSize) |
| Deserializes a camera model from a raw byte buffer. | |
| CameraModel | scaled (double scale) const |
| Returns a new camera model with all intrinsic parameters scaled by a given factor. | |
| CameraModel | roi (const cv::Rect &roi) const |
| Returns a new camera model adjusted for a given region of interest (ROI). | |
| cv::Mat | rectifyImage (const cv::Mat &raw, int interpolation=cv::INTER_LINEAR) const |
| Rectifies a raw image using the precomputed rectification maps. | |
| cv::Mat | rectifyDepth (const cv::Mat &raw) const |
| Rectifies a raw depth image using the precomputed rectification maps. | |
| void | project (float u, float v, float depth, float &x, float &y, float &z) const |
| Projects a 2D pixel and depth value into a 3D point in the camera coordinate frame (/camera_link). | |
| void | reproject (float x, float y, float z, float &u, float &v) const |
| Reprojects a 3D point in the camera frame (/camera_link) into 2D image coordinates (floating-point). | |
| void | reproject (float x, float y, float z, int &u, int &v) const |
| Reprojects a 3D point in the camera frame (/camera_link) into 2D image coordinates (rounded to int). | |
| bool | inFrame (int u, int v) const |
| Checks if a given pixel coordinate lies within the image bounds. | |
Static Public Member Functions | |
| static Transform | opticalRotation () |
| Returns the default optical rotation to convert image coordinates to robot coordinates. | |
Represents a pinhole camera model containing intrinsic and extrinsic parameters, used for projection, rectification, and transformation.
This class encapsulates camera calibration data, including intrinsic parameters (fx, fy, cx, cy), distortion coefficients, rectification and projection matrices. It provides utility functions for image rectification, projection from 2D to 3D, and vice versa.
This class supports the 4 to 14 parameters Radial Tangential distortion model (also called Plumb Bob or Brown-Conrady model) and 4 parameters Fish Eye model (also known as Equidistant model).
Definition at line 52 of file CameraModel.h.
| rtabmap::CameraModel::CameraModel | ( | const std::string & | name, |
| const cv::Size & | imageSize, | ||
| const cv::Mat & | K, | ||
| const cv::Mat & | D, | ||
| const cv::Mat & | R, | ||
| const cv::Mat & | P, | ||
| const Transform & | localTransform = opticalRotation() |
||
| ) |
Constructor using full camera parameters.
| name | Camera name or ID. |
| imageSize | Size of the image (width x height). |
| K | Intrinsic matrix (3x3). |
| D | Distortion coefficients, 1xN matrix where N is between 4 and 14 parameters: k1,k2,p1,p2[,k3[,k4,k5,k6[,s1,s2,s3,s4[,tx,ty]]]]). To set Fish Eye / Equidistant model, it is implicitly used if we provide 6 values like this: [k1,k2,0,0,k3,k4], where you only need to fill "k" parameters. |
| R | Rectification matrix (3x3). |
| P | Projection matrix (3x4). |
| localTransform | Local transform to apply to the camera frame. |
| rtabmap::CameraModel::CameraModel | ( | double | fx, |
| double | fy, | ||
| double | cx, | ||
| double | cy, | ||
| const Transform & | localTransform = opticalRotation(), |
||
| double | Tx = 0.0f, |
||
| const cv::Size & | imageSize = cv::Size(0, 0) |
||
| ) |
Minimal constructor using intrinsic parameters. This assumes the images are already rectified.
| fx | Focal length x. |
| fy | Focal length y. |
| cx | Principal point x. |
| cy | Principal point y. |
| localTransform | Local transform to apply to the camera frame. |
| Tx | Baseline * fx (optional). Mainly used in case of stereo pair. |
| imageSize | Image size (optional). |
|
inlinevirtual |
Destructor.
Definition at line 128 of file CameraModel.h.
|
inlinestatic |
Returns the default optical rotation to convert image coordinates to robot coordinates.
Image frame: x -> right, y -> down, z -> forward
Robot frame: x -> forward, y -> left, z -> up
Definition at line 63 of file CameraModel.h.
| bool rtabmap::CameraModel::initRectificationMap | ( | ) |
Initializes the rectification maps used to undistort and rectify images.
This function prepares the mapX_ and mapY_ lookup tables used for image rectification. It supports both standard radial-tangential distortion and fisheye/equidistant distortion models.
D_.cols == 6), it uses cv::fisheye::initUndistortRectifyMap() to create the rectification maps. This requires OpenCV ≥ 2.4.10.cv::initUndistortRectifyMap() for plumb bob or rational polynomial models.imageSize_ must be non-zero.D_ must be a 1-row matrix with an accepted number of columns (4, 5, 6, 8, 12, or 14).R_ must be a 3x3 rectification matrix.P_ must be a 3x4 projection matrix.true if the rectification maps were successfully initialized (mapX_ and mapY_ are not empty), false otherwise.
|
inline |
Checks if the rectification map has been initialized.
Definition at line 159 of file CameraModel.h.
|
inline |
Checks if the model is valid for 2D->3D projection.
Definition at line 164 of file CameraModel.h.
|
inline |
Checks if the model is valid for 3D->2D reprojection.
Definition at line 168 of file CameraModel.h.
|
inline |
Checks if the model has sufficient data for image rectification.
Definition at line 172 of file CameraModel.h.
|
inline |
Sets the camera name, used to set a camera name when saving to a file.
Definition at line 183 of file CameraModel.h.
|
inline |
Returns the camera name.
Definition at line 185 of file CameraModel.h.
|
inline |
Returns focal length in x.
Definition at line 188 of file CameraModel.h.
|
inline |
Returns focal length in y.
Definition at line 190 of file CameraModel.h.
|
inline |
Returns principal point x.
Definition at line 192 of file CameraModel.h.
|
inline |
Returns principal point y.
Definition at line 194 of file CameraModel.h.
|
inline |
Returns the x translation (usually fx * baseline in case of stereo, otherwise would be 0).
Definition at line 196 of file CameraModel.h.
|
inline |
Returns the raw intrinsic matrix (before rectification).
Definition at line 199 of file CameraModel.h.
|
inline |
Returns the raw distortion coefficients (before rectification).
Definition at line 201 of file CameraModel.h.
|
inline |
Returns the rectified camera intrinsic matrix if the projection matrix P exists, otherwise returns the raw intrinsic matrix.
Definition at line 203 of file CameraModel.h.
|
inline |
Returns the rectified distortion coefficients (1x5 filled with zeros) if the projection matrix P exists, otherwise returns the raw distortion coefficients.
Definition at line 205 of file CameraModel.h.
|
inline |
Returns the rectification matrix.
Definition at line 207 of file CameraModel.h.
|
inline |
Returns the projection matrix.
Definition at line 209 of file CameraModel.h.
|
inline |
Sets the local transform of the camera (base frame to optical frame).
Definition at line 212 of file CameraModel.h.
|
inline |
Returns the local transform (base frame to optical frame).
Definition at line 214 of file CameraModel.h.
| void rtabmap::CameraModel::setImageSize | ( | const cv::Size & | size | ) |
Sets the image size of the camera model and updates the principal point if undefined.
This function updates the internal image size (imageSize_) with the provided size. If the intrinsic matrices (K_ or P_) are present and the principal point coordinates (cx, cy) are zero, they are set to the image center (width/2 - 0.5, height/2 - 0.5).
This ensures the camera model remains valid and useful even when the calibration file has no principal point set or the image size is updated manually.
| size | The new image size. It must be either both dimensions zero (clearing) or both positive. |
size.width > 0 && size.height > 0 or size.width == 0 && size.height == 0 imageSize_, and adjusts cx and cy in K_ and P_ if they were initially zero.K_ or P_ are not initialized (empty()), no updates will be applied to them.
|
inline |
Definition at line 236 of file CameraModel.h.
|
inline |
Definition at line 237 of file CameraModel.h.
|
inline |
Definition at line 238 of file CameraModel.h.
| double rtabmap::CameraModel::fovX | ( | ) | const |
Returns the horizontal field of view (FoV) in radians.
The FoV is computed using the pinhole camera model as:
| double rtabmap::CameraModel::fovY | ( | ) | const |
Returns the vertical field of view (FoV) in radians.
The FoV is computed using the pinhole camera model as:
| double rtabmap::CameraModel::horizontalFOV | ( | ) | const |
Returns the horizontal field of view in degrees.
Converts the result of fovX() from radians to degrees.
| double rtabmap::CameraModel::verticalFOV | ( | ) | const |
Returns the vertical field of view in degrees.
Converts the result of fovY() from radians to degrees.
|
inline |
Checks if the distortion model is fisheye (6 coefficients: k1,k2,0,0,k3,k4).
Definition at line 283 of file CameraModel.h.
| bool rtabmap::CameraModel::load | ( | const std::string & | filePath, |
| bool | initRectificationMaps = true |
||
| ) |
Loads the camera model parameters from a YAML calibration file.
This method attempts to read camera intrinsic/extrinsic parameters and image size from a YAML file, typically in the ROS calibration format. If the distortion model is "fisheye" or "equidistant", we expect 4 coefficients, which are converted to a 6-coefficient format for internal representation.
Fields loaded (if present):
camera_name image_width (pixels)image_height (pixels)camera_matrix (K, 3x3 double matrix) distortion_coefficients (D, 1xN double matrix) distortion_model (string: name of the model)rectification_matrix (R, 3x3 double matrix) projection_matrix (P, 3x4 double matrix) local_transform (camera pose w.r.t robot frame)On success, the internal matrices and settings of the camera model are updated. If the model is valid for rectification, the rectification map is initialized.
| filePath | Absolute or relative path to the YAML file. |
| initRectificationMaps | Set to false to skip building the (potentially large) rectification maps when rectification won't be used (saves time and memory). |
| bool rtabmap::CameraModel::load | ( | const std::string & | directory, |
| const std::string & | cameraName, | ||
| bool | initRectificationMaps = true |
||
| ) |
Loads the camera model by constructing a file path from a directory and camera name.
This is a convenience wrapper around load(filePath) that constructs the file path as: directory + "/" + cameraName + ".yaml".
| directory | Path to the folder containing the camera YAML file. |
| cameraName | Base name of the camera file (without extension). |
| initRectificationMaps | Set to false to skip building the (potentially large) rectification maps when rectification won't be used (saves time and memory). |
| bool rtabmap::CameraModel::save | ( | const std::string & | directory | ) | const |
Saves the camera model parameters to a YAML calibration file in ROS format.
The file will include the following fields if they are not empty:
camera_nameimage_widthimage_heightcamera_matrix (K)distortion_coefficients (D)distortion_model (auto-detected based on number of distortion coefficients)rectification_matrix (R)projection_matrix (P)local_transform (camera pose w.r.t robot frame)If the distortion matrix contains 6 coefficients (used for fisheye), it is converted to a standard 4-coefficient format for ROS compatibility.
| directory | Path to the folder where the YAML file will be saved. |
name_ is empty, "camera.yaml" is used as the default filename. | std::vector< unsigned char > rtabmap::CameraModel::serialize | ( | ) | const |
Serializes the camera model to a binary format.
The serialization includes the camera intrinsics (K_, D_), rectification matrix (R_), projection matrix (P_), image size, and the local transform. The format is compact and suitable for file storage or transmission over a network.
Data layout:
double values), followed by float values for localTransformdeserialize().| unsigned int rtabmap::CameraModel::deserialize | ( | const std::vector< unsigned char > & | data | ) |
Deserializes a camera model from a byte vector.
This is a convenience wrapper around deserialize(const unsigned char*, unsigned int) that takes a std::vector<unsigned char> instead of a raw buffer.
| data | Byte vector containing data serialized by serialize(). |
| unsigned int rtabmap::CameraModel::deserialize | ( | const unsigned char * | data, |
| unsigned int | dataSize | ||
| ) |
Deserializes a camera model from a raw byte buffer.
Reads the camera intrinsics, distortion, rectification, projection matrices, image size, and local transform from a serialized binary format previously created with serialize().
| data | Pointer to the binary data buffer. |
| dataSize | Size of the data buffer in bytes. |
CameraModel::serialize(). Non-mono camera types are not supported. See StereoCameraModel to serialize/deserialize stereo models.| CameraModel rtabmap::CameraModel::scaled | ( | double | scale | ) | const |
Returns a new camera model with all intrinsic parameters scaled by a given factor.
This method scales the camera's intrinsic matrix (K_) and projection matrix (P_), as well as the image size, by the given scale factor. The distortion and rectification matrices are left unchanged.
Only valid camera models (i.e., those for which isValidForProjection() returns true) are scaled. If the model is invalid, a warning is issued and the original model is returned unchanged.
| scale | Scaling factor (> 0). For example, use 0.5 to downscale or 2.0 to upscale. |
| CameraModel rtabmap::CameraModel::roi | ( | const cv::Rect & | roi | ) | const |
Returns a new camera model adjusted for a given region of interest (ROI).
This method shifts the principal point (cx, cy) in the intrinsic matrix (K_) and projection matrix (P_) by subtracting the ROI’s top-left (x, y) offset. The image size is also set to the ROI size.
Only valid camera models (i.e., those for which isValidForProjection() returns true) can be adjusted. If the model is invalid, a warning is issued and the original model is returned unchanged.
| roi | Region of interest defined as a rectangle (typically a subwindow of the full image). |
| cv::Mat rtabmap::CameraModel::rectifyImage | ( | const cv::Mat & | raw, |
| int | interpolation = cv::INTER_LINEAR |
||
| ) | const |
Rectifies a raw image using the precomputed rectification maps.
This function applies geometric correction (rectification) to an image using the camera model's mapX_ and mapY_ rectification maps. It is typically used to correct lens distortion in images based on the calibration parameters.
| raw | Input raw image (e.g., from camera). Must be a valid cv::Mat. |
| interpolation | Interpolation method to use. Typically cv::INTER_LINEAR or cv::INTER_NEAREST. |
mapX_ and mapY_ must be initialized using initRectificationMap().| cv::Mat rtabmap::CameraModel::rectifyDepth | ( | const cv::Mat & | raw | ) | const |
Rectifies a raw depth image using the precomputed rectification maps.
This function applies geometric correction (rectification) to a 16-bit unsigned depth image. It performs a pixel-by-pixel bilinear interpolation, only if all neighboring pixels have valid (non-zero) depth values, and the variation among them is within 1% of their average.
The method is optimized to avoid introducing noise in regions of high depth variance.
| raw | Input raw depth image (CV_16UC1). Must contain 16-bit unsigned depth values. |
CV_16UC1, the function logs an error and returns a clone of the input.CV_16UC1. mapX_ and mapY_ must be initialized.| void rtabmap::CameraModel::project | ( | float | u, |
| float | v, | ||
| float | depth, | ||
| float & | x, | ||
| float & | y, | ||
| float & | z | ||
| ) | const |
Projects a 2D pixel and depth value into a 3D point in the camera coordinate frame (/camera_link).
This function uses the camera's intrinsic parameters to compute the 3D point corresponding to the given 2D image coordinates and depth value.
| u | Horizontal image coordinate (in pixels). | |
| v | Vertical image coordinate (in pixels). | |
| depth | Depth value at (u, v) in meters. | |
| [out] | x | Output X coordinate in 3D space. |
| [out] | y | Output Y coordinate in 3D space. |
| [out] | z | Output Z coordinate in 3D space (equals depth). |
depth <= 0, the output (x, y, z) will be set to NaN.| void rtabmap::CameraModel::reproject | ( | float | x, |
| float | y, | ||
| float | z, | ||
| float & | u, | ||
| float & | v | ||
| ) | const |
Reprojects a 3D point in the camera frame (/camera_link) into 2D image coordinates (floating-point).
This function computes the image plane coordinates for a given 3D point using the camera's intrinsic parameters.
| x | X coordinate in camera space. | |
| y | Y coordinate in camera space. | |
| z | Z coordinate in camera space (must be non-zero). | |
| [out] | u | Output horizontal image coordinate (float). |
| [out] | v | Output vertical image coordinate (float). |
z != 0| void rtabmap::CameraModel::reproject | ( | float | x, |
| float | y, | ||
| float | z, | ||
| int & | u, | ||
| int & | v | ||
| ) | const |
Reprojects a 3D point in the camera frame (/camera_link) into 2D image coordinates (rounded to int).
This version of reproject() returns integer pixel indices, computed from the 3D position.
| x | X coordinate in camera space. | |
| y | Y coordinate in camera space. | |
| z | Z coordinate in camera space (must be non-zero). | |
| [out] | u | Output horizontal image coordinate (integer pixel). |
| [out] | v | Output vertical image coordinate (integer pixel). |
z != 0| bool rtabmap::CameraModel::inFrame | ( | int | u, |
| int | v | ||
| ) | const |
Checks if a given pixel coordinate lies within the image bounds.
| u | Horizontal image coordinate (in pixels). |
| v | Vertical image coordinate (in pixels). |
true if the pixel is within the image dimensions, false otherwise.[0, width), [0, height)