ploteries.ndarray_data_handlers

Classes

NDArraySpec(dtype, shape)

RaggedNDArrayDataHandler(data_store, name[, ...])

Writes rows in data_records table that each contain one numpy array with arbitrary shape and dtype.

UniformNDArrayDataHandler(data_store, name)

Stores numpy arrays (and compatible, including scalars and lists thereof) efficiently enforcing the same shape and dtype, and loads them as a large numpy array, with each data_records table row corresponding to an entry (possibly multi-dimensional) in the numpy array.
class ploteries.ndarray_data_handlers.NDArraySpec(dtype, shape)
as_serializable()

Carries out the same function as TypeSerializer.as_serializable() but for the current object (self).

classmethod from_serializable(**kwargs)

Returns an object of this class. By default, calls __init__ with the provided keywords.

(See TypeSerializer.from_serializable())

class ploteries.ndarray_data_handlers.UniformNDArrayDataHandler(data_store: DataStore, name: str, ndarray_spec: NDArraySpec | None = None, _decoded_data_def=None)

Stores numpy arrays (and compatible, including scalars and lists thereof) efficiently enforcing the same shape and dtype, and loads them as a large numpy array, with each data_records table row corresponding to an entry (possibly multi-dimensional) in the numpy array. Supports lazy (upon first record add operation) definition of the ndarrary_spec (i.e., ndarray dtype and shape).

Supports multi-dimensional arrays of arbitary dtype (except Object) including (possibly nested) structured arrays. Supports also scalar inputs (e.g., integers, floats) and array-like inputs that can be passed to :method:`np.require` (e.g., lists, tuples).

Data store tables can be created in one of two ways:

1. By explicitly calling :method:`write_def`: ` und_dh = UniformNDArrayDataHandler(data_store, 'name', {'dtype':dtype('f'), 'shape':(10,1)}) und_dh.write_def() ` This method will fail if param ndarray_spec is not explicitly provided at initialization.

2. By adding data to an implicitly initialized object: ` und_dh = UniformNDArrayDataHandler(data_store, 'name') und_dh.add_data(0, np.array([0.0, 1.0])) ` This method can also be used with explicitly initialized objects, and a type-check will be carried out in this case.

3. Using from_name() ` und_dh = UniformNDArrayDataHandler.from_name(data_store, 'data_name') `

Parameters:

  • data_store – ploteries.data_store.DataStore object.

  • name – Data name.

  • ndarray_spec – An NDArraySpec producible.

data_store = None
decoded_data_def = None
property ndarray_spec

The dtype and shape of data in each record.

classmethod from_def_record(data_store, data_def_record)

Initializes a data handler object from a raw (encoded) record from the data store’s data_defs table.

encode_params(ndarray_spec=None)

Produces the params field to place in the data_defs record.

property row_num_bytes

Number of bytes in each record.

add_data(index: int, arr: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], connection=None)

Add new data row.

encode_record_bytes(arr)

Encodes the record’s data to bytes to be added to the 'bytes' field of the data_records table.

decode_record_bytes(data_bytes)

Decodes the record’s 'bytes' field to produce the record’s data.

merge_records_data(records_data)

Merges the list of decoded record bytes to create the 'data' field of the dictionary output by the load function.

class ploteries.ndarray_data_handlers.RaggedNDArrayDataHandler(data_store, name, _decoded_data_def=None, connection=None)

Writes rows in data_records table that each contain one numpy array with arbitrary shape and dtype. Supports multi-dimensional arrays of arbitary dtype (except Object) including (possibly nested) structured arrays.

Parameters:

  • data_store – ploteries.data_store.DataStore object.

  • name – Data name.

data_store = None
decoded_data_def = None
classmethod from_def_record(data_store, data_def_record)

Initializes a data handler object from a raw (encoded) record from the data store’s data_defs table.

encode_record_bytes(arr)

Encodes the record’s data to bytes to be added to the 'bytes' field of the data_records table.

decode_record_bytes(data_bytes)

Decodes the record’s 'bytes' field to produce the record’s data.

merge_records_data(records_data)

Merges the list of decoded record bytes to create the 'data' field of the dictionary output by the load function.