Base Class

Model of machine vision camera.

* NOTE: THIS IS CAMERA CLASS DEVELOPED FOR SOFAST, SHOULD BE MERGED WITH HELIO_SCAN VERSION OF CAMERA. *

class opencsp.common.lib.camera.Camera.Camera(intrinsic_mat: ndarray, distortion_coef: ndarray, image_shape_xy: tuple[int, int], name: str)

Bases: object

Calibrated machine vision camera representation. This represents a pinhole model of a camera/lens assembly and includes lens distortion through the use of distortion coefficients. This model uses the distortion model used by OpenCV; see https://docs.opencv.org/4.9.0/d9/d0c/group__calib3d.html for more information.

Parameters:

• intrinsic_mat (np.ndarray) – 3x3 numpy array, intrinsic camera matrix

• distortion_coef (np.ndarray) – 1d array, distortion coefficients, typically length 4.

• image_shape_xy (tuple(int)) – (x, y), image size in pixels.

• name (str) – Name of camera/lens combination.

__init__(intrinsic_mat: ndarray, distortion_coef: ndarray, image_shape_xy: tuple[int, int], name: str)

classmethod load_from_hdf(file: str)

Loads from HDF5 file

Parameters:

file (string) – HDF5 file to load

project(P_object: Vxyz, R_object_cam: Rotation, V_cam_object_cam: Vxyz) → Vxy

Projects points in 3D space to the camera sensor.

Parameters:

• P_object (Vxyz, float32 or float64) – 3D points to project, object coordinates.

• R_object_cam (Rotation) – Rotation from object to camera coordinates.

• V_cam_object_cam (Vxyz) – Translation from camera to object in camera coordinates.

Returns:

Projected points, pixels.

Return type:

Vxy.

project_mat(pts_object: ndarray, rot_vec: ndarray, v_cam_object_cam: ndarray) → ndarray

Identical to project but points in matrix form.

Parameters:

• pts_object (ndarray) – Nx3 3D points to project, object coordinates.

• rot_vec (ndarray) – Rotation vector from object to camera coordinates.

• v_cam_object_cam (ndarray) – 1D size (3,) translation vector from camera to object origin in camera coordinates.

Returns:

Nx2 array of projected points.

Return type:

ndarray

save_to_hdf(file: str)

Saves to HDF5 file

Parameters:

file (string) – HDF5 file to save

vector_from_pixel(pixels: Vxy) → Uxyz

Calculates pointing directions for given pixel

Parameters:

pixels (Vxy[float32 or float64]) – Pixel locations on camera sensor

Returns:

Poining direction for each input pixel

Return type:

Uxyz

property image_shape_yx

Image Acquisition Classes

class opencsp.common.lib.camera.ImageAcquisitionAbstract.ImageAcquisitionAbstract

Bases: ABC

Abstract base class for image acquisition from cameras.

This class defines the interface for acquiring images from various camera types. It implements a multiton design pattern to ensure that only one instance of a camera is active at a time. The class provides methods for exposure calibration, frame retrieval, and managing camera settings.

_instances

A dictionary of all instantiated camera instances, ensuring that each index corresponds to a unique camera instance.

Type:

dict[int, ImageAcquisitionAbstract]

_next_instance_idx

The index to use for the next camera instance added to the _instances dictionary.

Type:

int

on_close

A list of callback functions to be executed when the camera is closed.

Type:

list[Callable[[ImageAcquisitionAbstract], None]]

is_closed

A flag indicating whether the camera connection is closed.

Type:

bool

__init__()

calibrate_exposure()

Sets the camera’s exposure so that only 1% of pixels are above the set saturation threshold. Uses a binary search algorithm.

static cam_options() → dict[str, type[ImageAcquisitionAbstract]]

Defines camera objects to choose from (camera description, python type)

abstract close()

Closes the camera connection

abstract get_frame()

Gets a single frame from the camera

static instance(idx: int | None = None) → ImageAcquisitionAbstract | None

Get one of the global camera (ImageAcquisition) instances, if available.

If the camera has been closed, then this function will return None, even if the instance exists.

We use the multiton design pattern (several static global instances) for this class because we should only ever have one open instance per camera.

Parameters:

idx (int, None) – Which instance to return. If None, then the first non-closed instance is returned.

abstract instance_matches(possible_matches: list[ImageAcquisitionAbstract]) → bool

Returns true if there’s another instance in the list of possible matches that is equal to this instance. False if no other instances match.

Parameters:

possible_matches (list[ImageAcquisitionAbstract]) – The other cameras to match against. Does not include cameras that have been closed.

static instances(subclass: type[ImageAcquisitionAbstract] | None = None) → list[ImageAcquisitionAbstract]

Get all global camera (ImageAcquisition) instances, as available.

If a camera has been closed, then this function will not include it, even if the instance exists.

We use the multiton design pattern (several static global instances) for this class because we should only ever have one open ImageAcquisitionAbstract instance per camera.

Parameters:

subclass (type[ImageAcquisitionAbstract], optional) – Limits the type of camera instances returned. If None, then all available are returned.

abstract property exposure_time: int

Camera exposure_time value (microseconds)

property exposure_time_seconds: float

Camera exposure_time value (seconds)

abstract property frame_rate

Camera frame rate (units of FPS)

abstract property frame_size

camera frame size (X pixels by Y pixels)

abstract property gain

Camera gain value

abstract property max_value

Camera’s maximum saturation value. Example: If camera outputs 8-bit images, max_value = 255

abstract property shutter_cal_values: ndarray

Returns camera exposure_time values to use when calibrating the exposure_time. These values should be monotonically increasing from lowest shutter values to largest shutter values. The values need not be evely spaced, but the camera shutter is set to one of these values.

class opencsp.common.lib.camera.ImageAcquisition_DCAM_color.ImageAcquisition(instance: int = 0, pixel_format: str = 'BayerRG12')

Bases: ImageAcquisitionAbstract

__init__(instance: int = 0, pixel_format: str = 'BayerRG12')

Class to control a Basler DCAM color camera. Grabs one frame at a time. Assumes the color camera is in a Bayer pattern. The gain is initially set to the lowest possible value.

Parameters:

• instance (int) – The Nth instance of cameras (sorted by serial number) to instantiate.

• pixel_format (str) –

  Available formats include:

  • BayerRG12 (default)

  • BayerRG12Packed

  • Other RGB based formats as defined by Basler

close()

Closes the camera connection

get_frame(encode: bool = True) → ndarray

Captures a single frame from the camera.

This method initiates the frame capture process and retrieves the image data from the camera. The captured image can be returned in its raw Bayer format or converted to an RGB format based on the encode parameter.

Parameters:

encode (bool, optional) – If True (default), the captured Bayer-encoded image will be converted to a 3D RGB image using the encode_RG_to_RGB function. If False, the raw Bayer-encoded image will be returned.

Returns:

The captured image as a NumPy array. The shape of the array will depend on the encoding: - If encode is True, the shape will be (height, width, 3) for RGB images. - If encode is False, the shape will be (height, width) for Bayer images.

Return type:

np.ndarray

Raises:

pylon.RuntimeException – If the frame grab is unsuccessful, an exception is raised indicating the failure to capture the frame.

instance_matches(possible_matches: list[ImageAcquisitionAbstract]) → bool

Determine whether this camera instance matches any instance in the provided list.

This method checks if there is another camera instance in the possible_matches list that has the same basler_index as the current instance.

Parameters:

possible_matches (list[ImageAcquisitionAbstract]) – A list of camera instances to check against. Each instance should be of type ImageAcquisitionAbstract and may have a basler_index attribute.

Returns:

True when the first matching instance is found; False otherwise.

Return type:

bool

property exposure_time: int

Camera exposure_time value (microseconds)

property frame_rate: float

Camera frame rate (units of FPS)

property frame_size: tuple[int, int]

camera frame size (X pixels by Y pixels)

property gain: float

Camera gain value

property max_value: int

Camera’s maximum saturation value. Example: If camera outputs 8-bit images, max_value = 255

property shutter_cal_values: ndarray

Returns camera exposure_time values to use when calibrating the exposure_time. These values should be monotonically increasing from lowest shutter values to largest shutter values. The values need not be evely spaced, but the camera shutter is set to one of these values.

class opencsp.common.lib.camera.ImageAcquisition_DCAM_mono.ImageAcquisition(instance: int = 0, pixel_format: str = 'Mono8')

Bases: ImageAcquisitionAbstract

__init__(instance: int = 0, pixel_format: str = 'Mono8')

Class to control a Basler DCAM monochromatic camera. Grabs one frame at a time. The gain is initially set to the lowest possible value.

Parameters:

• instance (int) – The Nth instance of cameras (sorted by serial number) to instantiate.

• pixel_format (str) –

  Available formats include:

  • Mono8 (default)

  • Mono12

  • Mono12Packed

  • Others as defined by Basler

close()

Closes the camera connection

get_frame() → ndarray

Captures a single frame from the Basler DCAM monochromatic camera.

This method initiates the frame capture process and retrieves the image data from the camera. The method waits for the frame to be captured and returns the image data as a NumPy array.

Returns:

The captured image as a NumPy array. The shape of the array will depend on the pixel format set during initialization (e.g., Mono8, Mono12).

Return type:

np.ndarray

Raises:

pylon.RuntimeException – If the frame grab is unsuccessful, an exception is raised indicating the failure to capture the frame.

Notes

The method calculates the expected exposure time in milliseconds and uses this value to set a timeout for the frame retrieval. The timeout is set to 1.5 times the expected image acquisition time to ensure that the frame is captured successfully.

instance_matches(possible_matches: list[ImageAcquisitionAbstract]) → bool

Determine whether this camera instance matches any instance in the provided list.

This method checks if there is another camera instance in the possible_matches list that has the same basler_index as the current instance.

Parameters:

possible_matches (list[ImageAcquisitionAbstract]) – A list of camera instances to check against. Each instance should be of type ImageAcquisitionAbstract and may have a basler_index attribute.

Returns:

True when the first matching instance is found; False otherwise.

Return type:

bool

property exposure_time: int

Camera exposure_time value (microseconds)

property frame_rate: float

Camera frame rate (units of FPS)

property frame_size: tuple[int, int]

camera frame size (X pixels by Y pixels)

property gain: float

Camera gain value

property max_value: int

Camera’s maximum saturation value. Example: If camera outputs 8-bit images, max_value = 255

property shutter_cal_values: ndarray

Returns camera exposure_time values to use when calibrating the exposure_time. These values should be monotonically increasing from lowest shutter values to largest shutter values. The values need not be evely spaced, but the camera shutter is set to one of these values.

class opencsp.common.lib.camera.ImageAcquisition_MSMF.ImageAcquisition(instance: int = 0)

Bases: ImageAcquisitionAbstract

__init__(instance: int = 0)

close()

Closes the camera connection

get_frame() → ndarray

Captures a single frame from the connected camera.

This method reads a frame from the camera and returns it as a NumPy array. If the captured frame is in color (3-dimensional), it is converted to a grayscale image by averaging the color channels. The method raises an exception if the frame capture is unsuccessful.

Returns:

The captured image as a NumPy array. The shape of the array will be: - (height, width) for grayscale images. - If the input frame is in color, it will be converted to grayscale by averaging the channels.

Return type:

np.ndarray

Raises:

• Exception – If the frame was not captured successfully, an exception is raised indicating the failure to capture the frame.

• ValueError – If the output frame does not have 2 or 3 dimensions, a ValueError is raised indicating the incorrect number of dimensions.

instance_matches(possible_matches: list[ImageAcquisitionAbstract]) → bool

Determine whether this camera instance matches any instance in the provided list.

This method checks if there is another instance of the ImageAcquisition class in the possible_matches list. Since only one MSMF camera is supported, the method returns True if any instance of ImageAcquisition is found; otherwise, it returns False.

Parameters:

possible_matches (list[ImageAcquisitionAbstract]) – A list of camera instances to check against. Each instance should be of type ImageAcquisitionAbstract.

Returns:

True if a matching instance of ImageAcquisition is found; False otherwise.

Return type:

bool

property exposure_time: float

Camera exposure_time value (microseconds)

property frame_rate: float

Camera frame rate (units of FPS)

property frame_size: tuple[int, int]

camera frame size (X pixels by Y pixels)

property gain: float

Camera gain value

property max_value: int

Camera’s maximum saturation value. Example: If camera outputs 8-bit images, max_value = 255

property shutter_cal_values: ndarray

Returns camera exposure_time values to use when calibrating the exposure_time. These values should be monotonically increasing from lowest shutter values to largest shutter values. The values need not be evely spaced, but the camera shutter is set to one of these values.

UCamera Classes and Utility Functions

Model of machine vision camera.

* NOTE: THIS IS CURRENTLY TWO SEPARATE MODELS, DUE TO EVOLVING NEEDS DURING CODE DEVELOPMENT. * * WE NEED TO MERGE THESE, RECONCILING CALLERS. * * WE ALSO SHOULD INTEGRATE THIS SEAMLESSLY WITH CAMERA CALIBRATON ANALYSIS. * * NOTE: THIS CODE NEEDS TO BE MERGED WITH THE CAMERA CLASS FROM THE SOFAST CODE, ALSO LOCATED * * IN THIS DIRECTORY. *

class opencsp.common.lib.camera.UCamera.Camera(name, sensor_x_mm, sensor_y_mm, pixels_x, pixels_y, focal_lengths_mm)

Bases: CsvInterface

Model of a camera and its optical properties.

__init__(name, sensor_x_mm, sensor_y_mm, pixels_x, pixels_y, focal_lengths_mm)

static csv_header(delimeter=',')

Returns the CSV column headings for the camera data.

Parameters:

delimeter (str, optional) – The delimiter to use in the CSV header (default is ‘,’).

Returns:

A string containing the CSV column headings.

Return type:

str

classmethod from_csv_line(data_row: list[str])

Creates a Camera object from a CSV data row.

Parameters:

data_row (list of str) – A list containing the camera data in CSV format.

Returns:

A tuple containing the Camera object and any remaining data.

Return type:

tuple

max_focal_length()

Returns the maximum focal length of the camera.

Returns:

The maximum focal length in meters.

Return type:

float

to_csv_line(delimeter=',')

Converts the Camera object to a CSV line.

Parameters:

delimeter (str, optional) – The delimiter to use in the CSV line (default is ‘,’).

Returns:

A string representing the Camera object in CSV format.

Return type:

str

class opencsp.common.lib.camera.UCamera.RealCamera(name='Mavic Zoom', n_x=3840, n_y=2160, f_x=2868.1, f_y=2875.9, c_x=1920.0, c_y=1080.0, k_1=-0.024778, k_2=0.012383, p_1=-0.00032978, p_2=-0.0001401)

Bases: CsvInterface

Model of a camera and its intrinsic parameters, including distortion.

__init__(name='Mavic Zoom', n_x=3840, n_y=2160, f_x=2868.1, f_y=2875.9, c_x=1920.0, c_y=1080.0, k_1=-0.024778, k_2=0.012383, p_1=-0.00032978, p_2=-0.0001401)

Initializes the RealCamera object with specified parameters.

Parameters:

• name (str, optional) – String describing the camera and lens (default is ‘Mavic Zoom’).

• n_x (int, optional) – Number of pixels in the horizontal direction (default is 3840).

• n_y (int, optional) – Number of pixels in the vertical direction (default is 2160).

• f_x (float, optional) – Focal length in pixels in the x-direction (default is 2868.1).

• f_y (float, optional) – Focal length in pixels in the y-direction (default is 2875.9).

• c_x (float, optional) – Optical center x-coordinate in pixels (default is 1920.0).

• c_y (float, optional) – Optical center y-coordinate in pixels (default is 1080.0).

• k_1 (float, optional) – Radial distortion coefficient (default is -0.024778).

• k_2 (float, optional) – Radial distortion coefficient (default is 0.012383).

• p_1 (float, optional) – Tangential distortion coefficient (default is -0.00032978).

• p_2 (float, optional) – Tangential distortion coefficient (default is -0.0001401).

construct_frame_box_pq()

Constructs the frame box coordinates for the camera image.

Returns:

A list containing the minimum and maximum coordinates of the frame box.

Return type:

list of list of float

static csv_header(delimeter=',')

Returns the CSV column headings for the RealCamera data.

Parameters:

delimeter (str, optional) – The delimiter to use in the CSV header (default is ‘,’).

Returns:

A string containing the CSV column headings.

Return type:

str

classmethod from_csv_line(data_row: list[str])

Creates a RealCamera object from a CSV data row.

Parameters:

data_row (list of str) – A list containing the camera data in CSV format.

Returns:

A tuple containing the RealCamera object and any remaining data.

Return type:

tuple

image_frame_corners()

Returns the coordinates of the corners of the image frame.

Returns:

A list containing the coordinates of the four corners of the image frame.

Return type:

list of list of float

max_focal_length()

Returns the maximum focal length of the camera.

Returns:

The maximum focal length in pixels.

Return type:

float

to_csv_line(delimeter=',')

Converts the RealCamera object to a CSV line.

Parameters:

delimeter (str, optional) – The delimiter to use in the CSV line (default is ‘,’).

Returns:

A string representing the RealCamera object in CSV format.

Return type:

str

opencsp.common.lib.camera.UCamera.ideal_camera_normal()

Creates a RealCamera object for an ideal normal camera.

Returns:

A RealCamera object configured for an ideal normal camera.

Return type:

RealCamera

opencsp.common.lib.camera.UCamera.ideal_camera_telephoto()

Creates a RealCamera object for an ideal telephoto camera.

Returns:

A RealCamera object configured for an ideal telephoto camera.

Return type:

RealCamera

opencsp.common.lib.camera.UCamera.ideal_camera_wide_angle()

Creates a RealCamera object for an ideal wide angle camera.

Returns:

A RealCamera object configured for an ideal wide angle camera.

Return type:

RealCamera

opencsp.common.lib.camera.UCamera.mavic_zoom()

Creates a Camera object for the Mavic Zoom.

Returns:

A Camera object configured for the Mavic Zoom.

Return type:

Camera

opencsp.common.lib.camera.UCamera.real_mavic_zoom()

Creates a RealCamera object for the Real Mavic Zoom.

Returns:

A RealCamera object configured for the Real Mavic Zoom with specified calibration values.

Return type:

RealCamera

opencsp.common.lib.camera.UCamera.real_sony_alpha_20mm_still()

Creates a RealCamera object for the Real Sony Alpha 20mm (still images).

Returns:

A RealCamera object configured for the Real Sony Alpha 20mm (still) with specified calibration values.

Return type:

RealCamera

opencsp.common.lib.camera.UCamera.real_sony_alpha_20mm_video()

Creates a RealCamera object for the Real Sony Alpha 20mm (video).

Returns:

A RealCamera object configured for the Real Sony Alpha 20mm (video) with extrapolated calibration values.

Return type:

RealCamera

opencsp.common.lib.camera.UCamera.sony_alpha_20mm_landscape()

Creates a Camera object for the Sony Alpha 20mm (landscape orientation).

Returns:

A Camera object configured for the Sony Alpha 20mm (landscape).

Return type:

Camera

opencsp.common.lib.camera.UCamera.sony_alpha_20mm_portrait()

Creates a Camera object for the Sony Alpha 20mm (portrait orientation).

Returns:

A Camera object configured for the Sony Alpha 20mm (portrait).

Return type:

Camera

opencsp.common.lib.camera.UCamera.ultra_wide_angle()

Creates a Camera object for an ultra wide angle camera.

Returns:

A Camera object configured for an ultra wide angle lens.

Return type:

Camera

Utility Functions

opencsp.common.lib.camera.image_processing.encode_RG_to_RGB(image: ndarray) → ndarray

Converts raw data from a Bayer patterned camera to NxMx3 RGB image. For each 2x2 pixel block, simply averages green pixels and uses R and B pixels as-is.

Parameters:

image (np.ndarray) –

Input 2d image directly from RG Bayer pattern sensor. Pixels are arranged as:

R G … G B … . . …

Returns:

NxMx3 RGB image

Return type:

np.ndarray

opencsp.common.lib.camera.image_processing.highlight_saturation(image: ndarray, saturation_value: int | float) → ndarray

Highlights saturated pixels red. Image can be 2d or 3d, a 3d image is returned.

Parameters:

• image (np.ndarray) – 2D or 3D image array

• saturation_value (int | float) – Value that defines saturation in the image

Returns:

rgb – 3D image array converted to scaled float with range [0, 1].

Return type:

ndarray