class opencsp.common.lib.file.AbstractAttributeParser.AbstractAttributeParser

Bases: ABC

Class for parsing attributes related to specific data processing steps, for writing the context of that data processing to adjacent ‘attributes’ .txt files.

Inhereting classes need to implement the following methods:

def attributes_key(self) -> str:
    raise NotImplementedError()

def set_defaults(self, other: 'InheretingClassName'):
    raise NotImplementedError()

def has_contents(self) -> bool:
    raise NotImplementedError()

def parse_my_contents(self, file_path_name_ext: str, raw_contents: str, my_contents: any):
    raise NotImplementedError()

def my_contents_to_json(self, file_path_name_ext: str) -> any:
    raise NotImplementedError()

In each inheriting class’s file, after the class definition, be sure to also include:

MyAttributesClassName.RegisterClass()

See also:

AttributesManager.py

classmethod RegisterClass()

Registers this AbstractAttributeParser subclass with the AttributesManager, allowing it to be used for parsing and saving attributes files.

append_contents_for_writing(file_path_name_ext: str, contents: dict[str, any]) → dict[str, any]

Add the contents from this parser to be written to an attributes file. If has_contents() == False, then this parser will be skipped.

Parameters:

• file_path_name_ext (str) – The name of the file to be saved to. May be None.

• contents (dict[str,any]) – The current contents to be written out.

Returns:

The modified contents dict with the additional contents for this parser.

Return type:

dict[str,any]

abstract attributes_key() → str

Returns the key this class (or this instance) uses to index into the attributes file. This key should closely match the class name.

abstract has_contents() → bool

Returns True if there are contents that have been set for this instance. False to skip this instance when saving out an attributes file.

load(attributes_file_path_name_ext: str)

Loads and parses the given file, populating this instance with that file’s contents.

Raises:

json.decode.JSONDecodeError: – If parsing the given file fails

abstract my_contents_to_json(file_path_name_ext: str) → any

Prepare the contents from this parser to be written to an attributes file. If has_contents() == False, then this parser will be skipped.

_extended_summary_

Parameters:

file_path_name_ext (str) – The path to the attributes file. May be None.

Returns:

my_contents – The contents to be recorded. Should be json-ifiable.

Return type:

dict|list|str|float

parse_attributes_file(file_path_name_ext: str, raw_contents: str, json_contents: dict[str, any]) → dict[str, any]

Parse this attribute parser’s specific contents out of the given json_contents. Once this parser’s contents have been registered with this instance, then they can be removed from the dict and the modified dict returned.

Parameters:

• file_path_name_ext (str) – The path to the attributes file. May be None.

• raw_contents (str) – The unparsed string contents of the attributes file.

• json_contents (dict[str, any]) – The JSON interpretted representation of the attributes file.

Returns:

The modified json_contents, with this parser’s contents stripped out.

Return type:

dict[str, any]

abstract parse_my_contents(file_path_name_ext: str, raw_contents: str, my_contents: any)

Parse this attribute parser’s specific contents.

Parameters:

• file_path_name_ext (str) – The path to the attributes file. May be None.

• raw_contents (str) – The unparsed string contents of the attributes file.

• my_contents (any) – The JSON interpretted value for the attributes_key() in the raw_contents.

save(attributes_file_path_name_ext: str, overwrite=None)

Loads and parses the given file, then writes the file back with this instance’s contents.

Parameters:

• attributes_file_path_name_ext (str) – The name of the file to load from + save to.

• overwrite (bool, optional) – Whether to overwrite the existing file, if any. If None, then the existing file will be read first, then overwritten. By default None.

abstract set_defaults(other: AbstractAttributeParser)

Replaces this instance’s None-valued contents with non-None-valued contents from the given other of the same parser type.

class opencsp.common.lib.file.AttributesManager.AttributesManager(*parsers: AbstractAttributeParser)

Bases: object

A class for managing writing and reading attributes .txt files, such as those found next to saved images.

The purpose of an attributes file is to record the context of a file in a human-readable format (we use JSON for this format). For images, this might such things as the date the image was created, the operations that have been applied, the experiment it was part of, etc.

There is a global standard set of parsers that instances of this class are built with. Other AbstractAttributeReader implementations should register themselves with cls.RegisterClass() when their respective files are imported. These default parsers can be overridden with specific instances by creating a new instance of this manager (for example, a parser can be created with customized interpretation arguments).

Example of writing an attributes file:

# img = log_scale(equinox_image)
# img.save("5w1_9am.png")
exp_parser = ExperimentParser()
attr = AttributesManager(exp_parser)
exp_parser.name = "Spring Equinox Test 2023"
attr.write_attributes("5w1_9am.txt")

Example of reading an attributes file:

# img.load("5w1_9am.png")
attr = AttributesManager()
attr.load("5w1_9am.txt")
exp_parser: ExperimentParser = attr.get_parser(ExperimentParser)
title = exp_parser.title

__init__(*parsers: AbstractAttributeParser)

get_attributes_dict(attributes_file_path_name_ext: str | None = None)

Get the attributes for all this instance’s parsers that have contents.

Parameters:

attributes_file_path_name_ext (str, optional) – The name of the file to save the attributes to, by default None

Returns:

contents – The attributes contents, where the key represents a parser and the value represents that parser’s contents.

Return type:

dict[str, Any]

get_parser(parser_class: type[AbstractAttributeParser], error_on_not_found=True) → AbstractAttributeParser

Returns the parser instance matching the given parser_class.

Example:

attr = am.AttributesManager()
attr.load(image_name.replace(".png", ".txt"))
image_attr = attr.get_parser(iap.ImageAttributeParser)
print(f"Image {image_name} provides context for the experiment '{image_attr.experiment_name}'.")

load(attributes_file_path_name_ext: str)

Loads the attributes from the given file into the parsers for this instance. The attributes can then be retrieved from the desired parser with self.get_parser(), or from all parsers with self.parsers.

Raises:

• FileExistsError: – The given file does not exist.

• json.decoder.JSONDecodeError: – The given file is not in JSON format.

save(attributes_file_path_name_ext: str, overwrite=False)

Saves the attributes from this instance’s parsers into the given file.

Parameters:

• attributes_file_path_name_ext (str) – The path/name.ext of the file to save to.

• overwrite (bool, optional) – True to overwrite the attributes file if it already exists, by default False

Raises:

FileExistsError: – If the attributes file already exists and overwrite is False

set_parser(parser: AbstractAttributeParser)

Sets the given parser as the default parser for the given class.

property parsers

Returns the list of all parsers for this instance. Parsers that were passed in the constructor or in set_parser() are returned first.

class opencsp.common.lib.file.CsvColumns.CsvColumns(columns: dict[str, list[str | Pattern]])

Bases: object

A class to help parse CSV files with a tentative structure by finding column name matches.

This class allows for the definition of expected column names and their aliases, and provides methods to parse the header and data rows of a CSV file.

classmethod SimpleColumns(header_row: list[str])

Creates a CsvColumns instance from a simple header row.

This method initializes the columns using the header row as both the names and aliases.

Parameters:

header_row (list[str]) – A list of column names from the CSV header.

Returns:

An instance of CsvColumns initialized with the provided header row.

Return type:

CsvColumns

__init__(columns: dict[str, list[str | Pattern]])

Initializes the CsvColumns instance with the provided column definitions. Helps to parse csv files that have a tentative structure to them by finding column name matches.

Parameters:

columns (dict[str, list[str | re.Pattern]]) – The anticipated column names and their corresponding aliases or regex patterns.

Example

cols = cc.CsvColumns({
    'latitude': ['lat'],
    'datetime': ['UTC', 'localtime', re.compile(r"^dt")]
})
rows = ft.from_csv('Flight log', log_path, log_file_ext)
cols.parse_header(rows[0])
lat = float(rows[1][cols['latitude']])
dt = datetime.fromisoformat(rows[1][cols['datetime']])

parse_data_row(data_row: list[str], row_idx=-1)

Parses a data row and extracts values based on the matched column indices.

Parameters:

• data_row (list[str]) – A list of values from a single row of the CSV file.

• row_idx (int, optional) – The index of the row being parsed, used for logging. Defaults to -1.

Returns:

A dictionary mapping column names to their corresponding values from the data row.

Return type:

dict[str, str]

parse_header(header_row: list[str], error_on_not_found: bool | list[str] = True, ok_if_not_found: list[str] | None = None, alternatives: dict[str, list[str]] | None = None)

Parses the header row to find matches for the defined columns.

This method updates the column indices based on the header row and checks for any missing columns, logging warnings or raising errors as specified.

Parameters:

• header_row (list[str]) – A list of column names from the CSV header.

• error_on_not_found (bool | list[str], optional) – If True, raises an error for any missing columns. If a list, raises an error for columns in that list. Defaults to True.

• ok_if_not_found (list[str], optional) – A list of column names that are acceptable to be missing. Defaults to None.

• alternatives (dict[str, list[str]], optional) – A dictionary mapping column names to lists of alternative names. Defaults to None.

class opencsp.common.lib.file.CsvInterface.CsvInterface

Bases: object

Template class for storing data in comma-separated-value files. Inheriting classes include methods to convert to/from csv lines.

Required Methods

csv_header: Callable[[str],str]

Static method. Takes at least one parameter ‘delimeter’ and returns the string that represents the csv header.

to_csv_line: Callable[[str], str]

Takes at least one parameter ‘delimeter’ and returns the string that represents the instance of that class.

from_csv_line: Callable[[list[str]], tuple[Any,list[str]]]

Class method. Takes at least the parameter ‘data’ and returns an instance of the class and the remaining data strings.

Optional Methods

to_csv: Callable[[str,str,str,bool], None]

Optional. Takes the description, file_path, file_name_ext, and error_if_dir_not_exist parameters and writes the header and any rows to a file.

from_csv: Callable[[str,str,str], list[Any]]

Optional. Class method. Takes the description, file_path, and file_name_ext parameters and returns a list of class instances from that file.

static csv_header(delimeter=',') → str

Return a simple string which can be used as the header line in a csv file.

classmethod from_csv(description: str, file_path: str, file_name_ext: str)

Return N instances of this class from a csv file with a header and N lines.

Basic implementation of from_csv. Subclasses are encouraged to extend this method.

classmethod from_csv_line(data: list[str]) → tuple[CsvInterface, list[str]]

Construct an instance of this class from the pre-split csv line ‘data’. Also return any leftover portion of the csv line that wasn’t used.

to_csv(description: str, file_path: str, file_name: str, error_if_dir_not_exist: bool = True, rows: list[CsvInterface] | None = None, overwrite=False)

Create a csv file with a header and one or more lines (one line per contained instance if this is a collection of CsvInterface objects).

If rows is not None, then write all of the given rows with the to_csv_line() method. If rows is None, then attempt to use self.rows or self.rows() if it exists to write all the contained rows.

This is the basic implementation of to_csv. Subclasses are encouraged to extend this method.

to_csv_line(delimeter=',') → str

Return a string representation of this instance, to be written to a csv file. Does not include a trailing newline.

class opencsp.common.lib.file.SimpleCsv.SimpleCsv(description: str, file_path: str, file_name_ext: str)

Bases: object

A class for simple parsing of CSV files.

This class allows for reading a CSV file and provides methods to access the header, columns, and rows of the file in a structured manner.

__init__(description: str, file_path: str, file_name_ext: str)

Initializes the SimpleCsv instance and parses the CSV file.

Parameters:

• description (str) – A description of the file to be processed, or None to suppress output to stdout.

• file_path (str) – The path to the CSV file to be processed.

• file_name_ext (str) – The name and extension of the CSV file to be processed.

Raises:

• FileNotFoundError – If the specified CSV file does not exist.

• ValueError – If the CSV file is empty or improperly formatted.

Example

parser = scsv.SimpleCsv("example file", file_path, file_name_ext)
for row_dict in parser:
    print(row_dict)

get_columns()

Returns a list of column names from the CSV file.

Returns:

A list of column names extracted from the CSV header.

Return type:

list[str]

get_header()

Returns the header of the CSV file as a comma-separated string.

Returns:

A string representation of the header row of the CSV file.

Return type:

str

get_rows()

Returns the rows of the CSV file as a list of dictionaries.

Each dictionary corresponds to a row in the CSV file, with column names as keys.

Returns:

A list of dictionaries representing the rows of the CSV file.

Return type:

list[dict[str, str]]