SpotAnalysis

class opencsp.common.lib.cv.SpotAnalysis.SpotAnalysis(name: str, image_processors: list[AbstractSpotAnalysisImageProcessor], save_dir: str | None = None, save_overwrite=False)

Bases: Iterator[SpotAnalysisOperable]

Spot Analysis class for characterizing beams of light.

This is meant to be a general beam characterization tool that receives as input an image (or video) of a single beam, and generates annotations and numerical statistics as output. In general, there is an image processing step, an image analysis step, and a visualization step. Any number of processors and analyzers can be chained together to form a pipeline for evaluation.

Once defined, a SpotAnalysis instance can evaluate any number of input images with either the set_primary_images() or set_input_operables() methods.

A list of possible use cases for this class include:

BCS
BCS-based heliostat calibration
Vertical laser characterization
Laser beam spread characterization
2f returned spot analysis
Motion characterization video analysis
Wind BCS video trajectory analysis
Laser-based cross-check
Earth mover’s metric for comparisons (possible implementations (1) scipy has a method or (2) https://www.cs.cmu.edu/~efros/courses/LBMV07/Papers/rubner-jcviu-00.pdf)
Enclosed power

These are our expected outputs:

Annotated images
Visualizations (ex 2d or 3d heat-flux map, side-by-side comparisons)
Accompanying statistics INI files
Other details text file (heliostat name and location, date and time, heliostat operator, etc…)
Powerpoint slides

The features necessary to support these use cases include:

BCS target location (BcsLocatorImageProcessor)
Square target location (TargetBoardLocatorImageProcessor)
NULL image subtraction (NullImageSubtractionImageProcessor)
Ambient and gradient light subtraction (NOT IMPLEMENTED YET)
Lens correction (NOT IMPLEMENTED YET)
Intensity per pixel generation (NOT IMPLEMENTED YET)
Centroid calculation (a-h) (NOT IMPLEMENTED YET)
FWHM analysis (a,b,d,e,j) (NOT IMPLEMENTED YET)
Astigmatism identification (a-e,j) (NOT IMPLEMENTED YET)
Automatic beam identification (b) (NOT IMPLEMENTED YET)
Video-to-frame extraction (f,g) (ImagesStream)
Video streaming (f,g) (NOT IMPLEMENTED YET)
Networked image collection (b) (NOT IMPLEMENTED YET)
Color-aware image subtraction (c,d,e,h) (NOT IMPLEMENTED YET)
Color-aware light subtraction (c,d,e,h) (NOT IMPLEMENTED YET)
Time series analysis (f,g) (NOT IMPLEMENTED YET)
Laser placement and pointing angle as image metadata (e,h) (NOT IMPLEMENTED YET)
Laser placement and pointing angle as separate input (e,h) (NOT IMPLEMENTED YET)
Earth mover’s metric (i) (NOT IMPLEMENTED YET)
Multi-image comparison (f,g,i) (NOT IMPLEMENTED YET)
Intensity-to-power mapping (a,d,i,j) (NOT IMPLEMENTED YET)
Noise smoothing (ConvolutionImageProcessor)
Noise removal and quanitification (NOT IMPLEMENTED YET)
Multi-image averaging over time (a-e,h-j) (AverageByGroupImageProcessor)
Fiducial identification (e,18,21,23,24) (NOT IMPLEMENTED YET)
Screen/camera plane homography (TargetBoardLocatorImageProcessor)
Self-PnP and stabilization (StabilizationImageProcessor)
Annotate power envelopes (NOT IMPLEMENTED YET)
Orthorectify wrt. target (TargetBoardLocatorImageProcessor)
Orthorectify wrt. beam (NOT IMPLEMENTED YET)
Spot radius in mrad, at angle Beta relative to centroid (NOT IMPLEMENTED YET)
Spot radius in mrad, at a half angle relative to the heliostat location (NOT IMPLEMENTED YET)
Beam radius in mrad, at angle Beta relative to centroid (NOT IMPLEMENTED YET)
Beam radius in mrad, at a half angle relative to the heliostat location (NOT IMPLEMENTED YET)
Cropping (CroppingImageProcessor)
Over and under exposure detection (ExposureDetectionImageProcessor)
Tagging images as NULL images (NOT IMPLEMENTED YET)
Filters (gaussian, box, etc) (ConvoluationImageProcessor)
Peak value pixel identification (HotspotImageProcessor)
Logorithmic scaling (LogScaleImageProcessor)
False color visualization (ViewFalseColorImageProcessor)
Over/under exposure visualization (NOT IMPLEMENTED YET)
Image fill/inpainting (InpaintImageProcessor)

The inputs to support these features include:

primary image or images (set_primary_images())
reference image (7,8,22) (set_default_support_images())
null image (1,2,12,13,20) (set_default_support_images())
comparison image (18) (set_default_support_images())
background selection (2,11,13) (NOT IMPLEMENTED YET)
camera lens characterization (3) (NOT IMPLEMENTED YET)
image server network address (11) (NOT IMPLEMENTED YET)
laser color/wavelength (12,13) (NOT IMPLEMENTED YET)
camera color characterization (12,13) (NOT IMPLEMENTED YET)
laser placement and pointing angle (16) (NOT IMPLEMENTED YET)
intensity-to-power mapping (19) (NOT IMPLEMENTED YET)
multiple primary images, or a primary video (20,24) (set_primary_images())
fiducial definition and location (25) (NOT IMPLEMENTED YET)
manual 3D point identification from 2D images (25) (NOT IMPLEMENTED YET)

Example usage:

group_assigner = AverageByGroupImageProcessor.group_by_name(re.compile(r"(foo|bar)"))
group_trigger = AverageByGroupImageProcessor.group_trigger_on_change()

image_processors = {
    'Crop': CroppingImageProcessor(x1=200, x2=600, y1=100, y2=400),
    'AvgG': AverageByGroupImageProcessor(group_assigner, group_trigger),
    'Echo': EchoImageProcessor(),
    'Noiz': ConvolutionImageProcessor(kernel="box", diameter=3),
    'Ve3d': View3dImageProcessor(crop_to_threshold=20, max_resolution=(100, 100)),
    'Stat': PopulationStatisticsImageProcessor(min_pop_size=1, initial_min=0, initial_max=255),
    'Fclr': ViewFalseColorImageProcessor(),
}
pptx_processor = PowerpointImageProcessor(
    save_dir=outdir,
    save_name="processing_pipeline",
    processors_per_slide=[
        [image_processors['Noiz'], image_processors['Ve3d']],
        [image_processors['Fclr']],
    ],
)
image_processors_list = list(image_processors.values()) + [pptx_processor]

spot_analysis = sa.SpotAnalysis(
    experiment_name, image_processors_list, save_dir=outdir
)
spot_analysis.set_primary_images(images_path_name_exts)

for result in spot_analysis:
    pass

__init__(name: str, image_processors: list[AbstractSpotAnalysisImageProcessor], save_dir: str | None = None, save_overwrite=False)

Parameters:

name (str) – The name of this instance. For example, this could be one of the use cases listed above.
image_processors (list[AbstractSpotAnalysisImageProcessor]) – The list of processors that will be pipelined together and used to process input images.
save_dir (str) – If not None, then primary images will be saved to the given directory as a PNG after having been fully processed. Defaults to None.
save_overwrite (bool) – If True, then overwrite any existing images in the save_dir with the new output. Defaults to False.

process_next()

Attempts to get the next processed image and results data from the image processors pipeline.

Returns:: result – The processed primary image and other associated data. None if done processing.
Return type:: SpotAnalysisOperable | None

save_image(operable: SpotAnalysisOperable, save_dir: str | None = None, save_ext: str = 'png', also_save_supporting_images=True, also_save_attributes_file=True)

Saves the primary image from the given operable.

Parameters:

operable (SpotAnalysisOperable) – The primary image to be saved
save_dir (str, optional) – The directory to save the image to. Defaults to self.save_dir.
save_ext (str, optional) – The type to save the image as. Can include any standard image type, as well as “np” or “npy” for numpy. Defaults to png.
also_save_supporting_images (bool, optional) – If True, then any supporting images are saved next to the primary image with additional extensions to their names. Defaults to True.
also_save_attributes_file (bool, optional) – If True, then any attributes from the SpotAnalysisOperable will be saved next to the primary image as a .txt file in JSON format.

Returns:

image_path_name_ext – The path to the saved image, if self.save_dir != None. None if the image wasn’t saved, or if the image exists and save_overwrite is False, or if done processing.

Return type:

str | None

set_default_data(operable: SpotAnalysisOperable): Provides extra data for use during image processing, as a default for when the data is not otherwise from the input operables. Note that this does not include the primary or supporting images.

set_default_support_images(support_images: dict[ImageType, CacheableImage]): Provides extra support images for use during image processing, as a default for when the support images are not otherwise from the input operables. Note that this does not include the primary images or other data.

set_image_processors(image_processors: list[AbstractSpotAnalysisImageProcessor]): Update the image processors for this instance. This is typically done by passing in the image processors to the class constructor.

set_input_operables(input_operables: _SpotAnalysisOperablesStream | list[SpotAnalysisOperable] | Iterator[SpotAnalysisOperable])

Assigns primary and supporting images, and other necessary data, in preparation for process_next().

Notes

Only one of set_primary_images or set_input_operables() should be used to assign input images.

set_primary_images(images: list[str] | list[ndarray] | VideoHandler | ImagesStream)

Assigns the images of the spot to be analyzed, in preparation for process_next().

Parameters:: images (list[str] | list[np.ndarray] | vh.VideoHandler | ImagesStream) – The primary input images to be processed. Can be a list of images path/name.ext, list of images loaded into numpy arrays, a video handler instance set with a video to be broken into frames, or an images stream.

Notes

Only one of set_primary_images or set_input_operables() should be used to assign input images.

default_data: SpotAnalysisOperable: Other supporting data for processing input images. If not None, then all values here will be made available for processing as the default values.

default_support_images: dict[ImageType, CacheableImage]: Other supporting images for processing input images. If not None, then all values here will be made available for processing as the default values.

image_processors: list[AbstractSpotAnalysisImageProcessor]: List of processors, one per step of the analysis. The output from each processor can include one or more operables, and is made available to all subsequent processors.

input_stream: _SpotAnalysisOperablesStream: The images to be processed.

name: The name of this instance. For example, this could be one of the use cases listed above.

save_dir: str: If not None, then primary images will be saved to the given directory as a PNG after having been fully processed.

save_idx: int: The counter used to create unique image file names to save to, as necessary.

save_overwrite: If True, then overwrite any existing images in the save_dir with the new output. Defaults to False.

saved_names: set[str]: The names of image files saved to, to ensure unique image file names are used.

visualization_coordinator: Manages interactive viewing of all visualization image processors.

Data Classes

These are the book keeping classes that enable the spot analysis pipeline.

class opencsp.common.lib.cv.spot_analysis.SpotAnalysisOperable.SpotAnalysisOperable(primary_image: ~opencsp.common.lib.cv.CacheableImage.CacheableImage, primary_image_source_path: str | None = None, supporting_images: dict[~opencsp.common.lib.cv.spot_analysis.ImageType.ImageType, ~opencsp.common.lib.cv.CacheableImage.CacheableImage] = <factory>, visualization_images: dict[AbstractSpotAnalysisImageProcessor, list[~opencsp.common.lib.cv.CacheableImage.CacheableImage]] = <factory>, _visualization_image_no_axes: ~opencsp.common.lib.cv.CacheableImage.CacheableImage | None = None, algorithm_images: dict[AbstractSpotAnalysisImageProcessor, list[~opencsp.common.lib.cv.CacheableImage.CacheableImage]] = <factory>, previous_operables: tuple[list[SpotAnalysisOperable], AbstractSpotAnalysisImageProcessor] | tuple[None, None] = (None, None), given_fiducials: list[~opencsp.common.lib.cv.fiducials.AbstractFiducials.AbstractFiducials] = <factory>, found_fiducials: list[~opencsp.common.lib.cv.fiducials.AbstractFiducials.AbstractFiducials] = <factory>, annotations: list[~opencsp.common.lib.cv.annotations.AbstractAnnotations.AbstractAnnotations] = <factory>, camera_intrinsics_characterization: any | None = None, light_sources: list[~opencsp.common.lib.csp.LightSource.LightSource] = <factory>, population_statistics: ~opencsp.common.lib.cv.spot_analysis.SpotAnalysisPopulationStatistics.SpotAnalysisPopulationStatistics | None = None, image_processor_notes: list[tuple[str, list[str]]] = <factory>, x_coordinates_transform: ~sympy.core.expr.Expr | None = None, y_coordinates_transform: ~sympy.core.expr.Expr | None = None)

Bases: object

Contains a list of all the pieces that might be used or generated during spot analysis. This helps to guarantee that related data stays together for the whole process. Note that although not all fields will be populated, the primary_image at least should be.

This class should require very little memory, as there can be a great many instances of this class in flight at any given time.

get_all_images(primary=True, supporting=True, visualization=True, algorithm=True) → list[CacheableImage]

Get a list of all images tracked by this operable including all primary images, supporting images, visualization, and algorithm images. Does not include images from previous operables.

Parameters:

primary (bool, optional) – True to include the primary image in the list of returned images. By default True.
supporting (bool, optional) – True to include the supporting images, if any, in the list of returned images. By default True.
visualization (bool, optional) – True to include the visualization images in the list of returned images. By default True.
algorithm (bool, optional) – True to include the algorithm images, if any, in the list of returned images. By default True.

Returns:

The images tracked by this operable.

Return type:

list[CacheableImage]

get_fiducials_by_type(fiducial_type: type[AbstractFiducials]) → list[AbstractAnnotations | AbstractFiducials]: Returns all fiducials from self.given_fiducials, self.found_fiducials, and self.annotations that match the given type.

get_primary_path_nameext() → tuple[str, str]

Finds the best source path/name.ext for the primary image of this operable.

The source path is chosen from among the primary_image’s .source_path, the primary_image_source_path, and the primary_image’s .cache_path_name_ext.

Returns:

source_path (str) – The path component of the source of the primary_image. “unknown_path” if there isn’t an associated source.
source_name_ext (str) – The name.ext component of the source of the primary_image. “unknown_image” if there isn’t an associated source.

is_ancestor_of(other: SpotAnalysisOperable) → bool: Returns true if this operable is in the other operable’s previous_operables tree. Does not check if this operable is the other operable.

replace_use_default_values(supporting_images: dict[ImageType, CacheableImage] | None = None, data: SpotAnalysisOperable | None = None) → SpotAnalysisOperable: Sets the supporting_images and other data for an operable where they are None for this instance. Returns a new operable with the populated values.

transform_coordinates(xy_pixels: Pxy) → tuple[bool, Pxy]

Given a set of (x,y) pixel coordinates, get the (x,y) location in real-world meters. If self.xy_coordinates_transform is None, then the input pixel value will be returned.

Parameters:

xy_pixels (Pxy) – The pixel coordinates to be transformed.

Returns:

transformed (bool) – True if xy_meters is in meters, False if it is in pixels.
xy_meters (Pxy) – The transformed coordinates in meters, or the input xy_pixels.

algorithm_images: dict[AbstractSpotAnalysisImageProcessor, list[CacheableImage]]: The images produced by the image processors to explain how certain values were determined.

See also

supporting_images, visualization_images

annotations: list[AbstractAnnotations]: The identified annotations in the currently processing image.

property best_primary_nameext: str: The name.ext of the source of the primary_image.

property best_primary_pathnameext: str: The path/name.ext of the source of the primary_image.

camera_intrinsics_characterization: any = None: Distortion, color, bit depth, etc of the camera. Maybe also distortion properties of the lens system.

found_fiducials: list[AbstractFiducials]: The identified fiducials in the currently processing image.

given_fiducials: list[AbstractFiducials]: Any fiducials handed to us in the currently processing image.

image_processor_notes: list[tuple[str, list[str]]]: Notes from specific image processors. These notes are generally intended for human use, but it is recommended that they maintain a consistent formatting so that they can also be used programmatically.

light_sources: list[LightSource]: The sources that produced the light that landed on the observed receiver surface. This can be used to indicate laser color, light intensity, or with simulations to provide a starting estimate of beam shape.

property max_popf: ndarray[Any, dtype[float64]]: Returns the maximum population float value, if it exists. Otherwise returns the maximum value for this instance’s primary image.

property min_popf: ndarray[Any, dtype[float64]]: Returns the minimum population float value, if it exists. Otherwise returns the minimum value for this instance’s primary image.

population_statistics: SpotAnalysisPopulationStatistics = None: The population statistics, as populated by PopulationStatisticsImageProcessor.

previous_operables: tuple[list[SpotAnalysisOperable], AbstractSpotAnalysisImageProcessor] | tuple[None, None] = (None, None): The operable(s) that were used to generate this operable, and the image processor that they came from, if any. If this operable has no previous operables registered with it, then this will have the value (None, None). Does not include do-nothing image processors such as EchoImageProcessor or SaveToFileImageProcessor.

primary_image: CacheableImage: The input image to be processed, or the output processed image.

primary_image_source_path: str = None: The path_name_ext of the primary image, if the source is a file on disk. This should be used as the secondary source of this information, after best_primary_pathnameext() and primary_image.source_path.

supporting_images: dict[ImageType, CacheableImage]: The supporting images, if any, that were provided with the associated input primary image. These images will be used as part of the computation.

See also

algorithm_images, visualization_images

visualization_images: dict[AbstractSpotAnalysisImageProcessor, list[CacheableImage]]: The images produced by the image processors as they operate on this instance. These images are used for visualization, debugging, and automatic powerpoint generation.

See also

algorithm_images, supporting_images

x_coordinates_transform: Expr = None: The expression to convert from (x,y) image values to real-world x coordinates, in meters. If None, then the input x value will be returned.

y_coordinates_transform: Expr = None: The expression to convert from (x,y) image values to real-world y coordinates, in meters. If None, then the input y value will be returned.

Bases: Iterator[CacheableImage]

A one-time iterator over a list of images.

Iterates over the given images. The next() method returns the next item from the input list.

Note that calling iter() on this instance DOES NOT restart iteration. This is to maintain the interface for streamable image sources, such as webcams or networked cameras, which have no replay ability.

Note that this does NOT build on the python asyncio stream library for networking. Such a class might be implemented later and will likely be called “ImagesStreamOverIP”.

Parameters:: images (Callable[[int],CacheableImage] | list[str|CacheableImage] | vh.VideoHandler | Iterator[str|CacheableImage]) – The images to iterate over.

class opencsp.common.lib.cv.spot_analysis.SpotAnalysisImagesStream.SpotAnalysisImagesStream(primary_iterator: ImagesIterable | ImagesStream, other_iterators: dict[ImageType, ImagesIterable | ImagesStream] | None = None)

Bases: Iterator[dict[ImageType, CacheableImage]]

This class combines the image streams for several SpotAnalysisImageTypes into one convenient package. This helps to guarantee that images that are supposed to be processed together stay together for the entirety of the SpotAnalysis pipeline.

__init__(primary_iterator: ImagesIterable | ImagesStream, other_iterators: dict[ImageType, ImagesIterable | ImagesStream] | None = None)

Parameters:

primary_iterator (ImagesIterable | ImagesStream) – The images of the spot to be analyzed.
other_iterators (dict[SpotAnalysisImageType,ImagesStream], optional) – Supporting images that provide extra information for spot analysis. Default None

class opencsp.common.lib.cv.spot_analysis.ImageType.ImageType(value)

Bases: Enum

The category of a single CacheableImage that is being passed through the image processing pipeline. Most images will be PRIMARY images, meaning that they will be the data being analyzed. Images of different types are grouped together by a SpotAnalysisImagesStream or SpotAnalysisOperable.

PRIMARY = 1: The image we are trying to analyze.

REFERENCE = 2: Contains a pattern to be compared or matched with in the PRIMARY image.

NULL = 3: The same as the PRIMARY image, but without a beam on target. Likely this will be used to subtract out the background.

COMPARISON = 4: For multi-image comparison, such as for re-alignment to a previous position, motion characterization, or measuring wind effect.

BACKGROUND_MASK = 5: A boolean image that indicates which pixels should be included in a computation (True to include, False to exclude).

VISUALIZATION = 6: Images used to view the results or state of image processing.

ALGORITHM = 7: Explainer images that visualize how an image processing step was completed.

static ALL() → tuple[ImageType]