ewoksid02.tasks.id02processingtask.ID02ProcessingTask#

class ewoksid02.tasks.id02processingtask.ID02ProcessingTask(inputs=None, varinfo=None, node_id=None, node_attrs=None, execinfo=None, profile_directory=None)[source]#

Bases: Task

This class contains processing support methods and saving methods in the ID02 SAXS pipeline. It extends the ID02LoopTask class and provides additional functionality for handling metadata, processing flags, and saving processed data to HDF5 files.This class is designed to be used as part of the ID02 pipeline.It does not contain a process method, that has to be implemented in the child class.

Optional Inputs:

  • detector_name (str): Name of the detector used for data acquisition. This is the only mandatory input.

  • scan_memory_url (str): URL for accessing scan memory in online processing.

  • beacon_host (str): Host and port to plug blissdata to the correct beacon server. Only for online processing.

  • reading_node (bool): Flag to indicate if the task should read data from the node.

  • filename_data (str): Path to the dataset file (Master file, Nexus writer) for offline processing.

  • filename_lima (str): Path to the first Lima file, the only place where some detector metadata can be found.

  • scan_nb (int): Scan number for identifying the dataset.

  • headers (dict): Only for Online processing. Dictionary containing headers information.

  • max_slice_size (int): Maximum number of frames to process in one iteration. Default is 20.

  • dataset_signal (numpy.ndarray): Signal dataset to be processed.

  • dataset_variance (numpy.ndarray): Variance dataset to be processed.

  • dataset_sigma (numpy.ndarray): Sigma dataset to be processed.

  • datatype (str): Datatype to be used to save the 2D data. Default and recommended is float32.

  • lima_url_template (str): Format string to locate the Lima file and the path to the data inside that file.

  • lima_url_template_args (dict): Dictionary to format the lima_url_template.

  • log_level (str): Logging level for the task. Default is “warning”.

  • processing_filename (str): Full path to the (new) output file.

  • processing_subtitle (str): Additional subtitle for the processing task.

  • subtitle (str): Subtitle for the processing task to be added to the output filename.

  • do_process (bool): Flag to enable or disable processing. Default is True.

  • do_save (bool): Flag to enable or disable saving of processed data. Default is True.

  • save_variance (bool): Flag to enable or disable saving of variance dataset. Default is False.

  • save_sigma (bool): Flag to enable or disable saving of sigma dataset. Default is True.

  • save_metadata (bool): Flag to enable or disable saving of metadata. Default is True.

  • last_index_read (int): Index of the last frame read in the dataset. Default is 0.

  • range_index_read (list): Range of indices to read from the dataset. This parameter is not propagated to the next task.

  • loop_nb (int): Current loop iteration number. Default is 0.

  • info (dict): Additional metadata to save.

  • info_history (dict): Additional metadata to propagate and save, creating a history of processing.

  • gc_collect (bool): Manually collect garbage at the end of every task.

  • lima_index_number_format (str): format to find the first Lima file (02%d by default)

Outputs:

  • last_index_read (int): Updated index of the last frame read.

  • loop_nb (int): Updated loop iteration number.

  • dataset_signal (numpy.ndarray): Processed signal dataset.

  • dataset_variance (numpy.ndarray): Processed variance dataset.

  • dataset_sigma (numpy.ndarray): Processed sigma dataset.

  • continue_pipeline (bool): Flag to indicate whether the pipeline should continue.

  • info_history (dict): Additional metadata to propagate and save, creating a history of processing.

Parameters:

  • inputs (Optional[Mapping])

  • varinfo (Optional[dict])

  • node_id (Union[str, int, tuple, None])

  • node_attrs (Optional[dict])

  • execinfo (Optional[dict])

  • profile_directory (Optional[dict])

MISSING_DATA = <MISSING_DATA>#
assert_ready_to_execute()#
cancel()#

Function called when a task is cancelled. To be implemented by the derived classes

property cancelled: bool#

Return True if the task has been cancelled by the user

classmethod class_nonce()#
classmethod class_nonce_data()#
classmethod class_registry_name()#
Return type:

Optional[str]

cleanup_references()#

Removes all references to the inputs. Side effect: fixes the uhash of the task and outputs

create_id02_processed_file(filename)[source]#
Parameters:

filename (str)

property done: bool#

Completed (with or without exception)

property exception: Exception | None#
execute(force_rerun=False, raise_on_error=True, cleanup_references=False)#
Parameters:

  • force_rerun (Optional[bool])

  • raise_on_error (Optional[bool])

  • cleanup_references (Optional[bool])

property failed: bool#

Completed with exception

fix_uhash()#

Fix the uhash when it is derived from the uhash data.

get_HS32C_array()[source]#
get_HS32V_array()[source]#
get_exposuretime_values()[source]#
Return type:

tuple

get_input_uhashes()#
get_input_value(key, default=<MISSING_DATA>)#
Parameters:

default (Any)

Return type:

Any

get_input_values()#
get_memory_info()[source]#
get_named_input_values()#
get_output_transfer_data()#

The values are either DataUri or Variable

get_output_uhashes()#
get_output_value(key, default=<MISSING_DATA>)#
Parameters:

default (Any)

Return type:

Any

get_output_values()#
get_parameter(key, to_integer=False, default=None)[source]#
Parameters:

  • key (str)

  • to_integer (bool)

get_positional_input_values()#
get_processing_parameters()[source]#
Return type:

dict

get_stream(index_pin=None, name=None, header_key_pin=None, prefix='', suffix='', subscan_2=False)[source]#
Parameters:

  • index_pin (int)

  • name (str)

  • header_key_pin (str)

  • prefix (str)

  • suffix (str)

  • subscan_2 (bool)

Return type:

tuple

get_stream_deltatime()[source]#
Return type:

tuple

get_stream_exposuretime()[source]#

In blissdata, this stream is (normally) called scalers:time (in subscan1) In the header, there is key “HSTime” pointing to the string ‘time’

Return type:

tuple

get_stream_exposuretime_raw()[source]#

In blissdata, this stream is (normally) called mcs:time_raw (in subscan1) In the header, there is key “HSTime” pointing to the string ‘time’

Return type:

tuple

get_stream_fast_timer()[source]#
Return type:

tuple

get_stream_monitor_0()[source]#
Return type:

tuple

get_stream_monitor_1()[source]#
Return type:

tuple

get_stream_slow_timer()[source]#
Return type:

tuple

get_streams_HS32_raw()[source]#
Return type:

list

get_streams_HS32_scalers()[source]#
Return type:

list

classmethod get_subclass(registry_name, _second_attempt=False)#

Retrieving a derived class

classmethod get_subclass_names()#
Return type:

List[str]

classmethod get_subclasses()#
get_uhash_init(serialize=False)#
classmethod input_model()#
Return type:

Optional[Type[BaseInputModel]]

classmethod input_names()#
Return type:

Set[str]

property input_uhashes#
property input_values#

DEPRECATED

property input_variables: VariableContainer#
property inputs: ReadOnlyVariableContainerNamespace#
instance_nonce()#
classmethod instantiate(registry_name, **kw)#

Factory method for instantiating a derived class.

Parameters:

  • registry_name (str) – for example “tasklib.tasks.MyTask” or “MyTask”

  • **kw – Task constructor arguments

  • registry_name

Returns Task:

is_first_loop()[source]#
is_new_cycle()[source]#
is_new_workflow()[source]#
property is_ready_to_execute#
property job_id: str | None#
property label: str#
load_scan()[source]#
log_allocated_memory()[source]#
log_debug(msg)[source]#
log_error(msg)[source]#
log_info(msg)[source]#
log_warning(msg)[source]#
property missing_inputs: VariableContainerMissingNamespace#
property missing_outputs: VariableContainerMissingNamespace#
property n_positional_inputs: int#
classmethod n_required_positional_inputs()#
Return type:

int

property named_input_values#

DEPRECATED

property node_id: str | int | tuple#
property npositional_inputs#

DEPRECATED

open_id02_file(filename, mode='r')[source]#
Parameters:

  • filename (str)

  • mode (str)

classmethod optional_input_names()#
Return type:

Set[str]

property output_metadata: dict | None#
classmethod output_model()#
Return type:

Optional[Type[BaseOutputModel]]

classmethod output_names()#
Return type:

Set[str]

property output_transfer_data#

DEPRECATED

property output_uhashes#

DEPRECATED

property output_values#

DEPRECATED

property output_variables: VariableContainer#
property outputs: VariableContainerNamespace#
property positional_input_values#

DEPRECATED

process()[source]#
read_from_stream_interpolate(stream_sliceable, stream_slice=None, datatype='float32')[source]#

stream_object is a sliceable object (numpy.ndarray or blissdata stream object) Valid for all streams, from subscan1 and subscan2

Parameters:

datatype (str)

Return type:

ndarray

classmethod required_input_names()#
Return type:

Set[str]

reset_state()#
run()[source]#

To be implemented by the derived classes

save()[source]#
set_uhash_init(pre_uhash=None, instance_nonce=None)#
Parameters:

  • pre_uhash (Union[str, bytes, UniversalHash, HasUhash, None])

  • instance_nonce (Optional[Any])

property succeeded: bool#

Completed without exception and with output values

property task_identifier: str#
property uhash: UniversalHash | None#
uhash_randomize()#
undo_fix_uhash()#
undo_randomize()#
property workflow_id: str | None#
On this page

This Page