scaling.shmutils

Shared Memory Utils module

class gridr.scaling.shmutils.SharedMemoryArray(shape, dtype, name, array_slice=None)

A class handler for managing shared memory buffers and their associated NumPy array views.

This class simplifies the creation, loading, and cleanup of NumPy arrays backed by Python’s multiprocessing.shared_memory module, enabling efficient data sharing between processes.

COUNTER

A class-level counter used for generating unique shared memory names.

Type:

int

shape

The shape of the NumPy array that will reside in shared memory.

Type:

tuple of int

dtype

The data type of the NumPy array elements.

Type:

numpy.dtype

name

The unique name of the shared memory segment.

Type:

str

array_slice

A tuple of slice objects to apply to the array after loading, providing a view of a sub-section of the shared memory.

Type:

tuple of slice, optional

smh

The handler for the shared memory buffer. None if not yet created or loaded, or after close().

Type:

shared_memory.SharedMemory or None

array

The NumPy array view onto the shared memory buffer. None if not yet created or loaded, or after close().

Type:

numpy.ndarray or None

classmethod build_sma_name(prefix)

Generates a supposedly unique name for a shared memory segment.

The name is constructed using a class-level counter, an optional prefix, the current timestamp, and a UUID4 string to maximize uniqueness. The class counter is incremented with each call.

Parameters:

prefix (str, optional) – An optional string prefix to include in the generated name. Defaults to None, resulting in an empty prefix.

Returns:

A unique string suitable for use as a shared memory segment name. Example: “1-my_prefix-202310-2715-3000-abcdef12-3456-7890-abcd-ef1234567890”

Return type:

str

classmethod clear_buffers(buffer_names)

Clears (unlinks) a list of named shared memory buffers.

This method iterates through a list of shared memory names and attempts to unlink (delete) each corresponding shared memory segment from the operating system. This effectively cleans up shared memory resources.

Parameters:

buffer_names (list of str) – A list of unique names of the shared memory buffers to be unlinked.

Return type:

NoReturn

classmethod clone(sma, **override)

Creates a new SharedMemoryArray instance by cloning an existing one.

This class method allows for creating a new SharedMemoryArray object with attributes copied from another SharedMemoryArray instance. Specific attributes can be overridden by providing keyword arguments.

Parameters:

• sma (SharedMemoryArray) – The SharedMemoryArray instance to clone. Its shape, dtype, name, and array_slice attributes will be used as defaults.

• **override – Keyword arguments to override any of the cloned attributes. For example: name=”new_name”, shape=(10, 20).

Returns:

A new SharedMemoryArray instance with the cloned or overridden attributes.

Return type:

SharedMemoryArray

close()

Closes the connection to the shared memory buffer.

This method releases the current process’s view of the shared memory segment. It does not unlink (delete) the shared memory segment itself; it only closes the current process’s connection. After calling close(), self.smh and self.array are set to None.

Return type:

NoReturn

create()

Creates the shared memory buffer and associates a NumPy array view.

This method allocates a shared memory segment with the specified name and size (derived from shape and dtype), then creates a NumPy array view that points to this shared memory. The array_slice attribute is not applied during creation; it’s used when the array is loaded (e.g., by another process, or via the load() method).

Return type:

NoReturn

load()

Loads the object by connecting to a previously created shared memory buffer.

This method attempts to connect to an existing shared memory segment identified by self.name. It then creates a NumPy array view onto this shared memory. If self.array_slice is defined, it applies this slice to the array, providing a view to a specific region.

Return type:

NoReturn

gridr.scaling.shmutils.create_and_register_sma(shape, dtype, register, prefix=None)

Creates a SharedMemoryArray and registers its name in a list.

This helper function simplifies the process of creating a new shared memory array. It first generates a unique name for the shared memory segment, then instantiates and creates the SharedMemoryArray object, and finally adds the generated shared memory name to a provided list for tracking (e.g., for later cleanup).

Parameters:

• shape (tuple of int) – The desired shape of the NumPy array to be created in shared memory.

• dtype (numpy.dtype) – The data type of the NumPy array elements.

• register (list of str) – A list to which the generated unique name of the shared memory segment will be appended. This list is typically used to keep track of active shared memory buffers for later management or clearing.

• prefix (str, optional) – An optional string prefix to use when generating the unique name for the shared memory segment. Defaults to None.

Returns:

The newly created and initialized SharedMemoryArray object.

Return type:

SharedMemoryArray

gridr.scaling.shmutils.deprecated(message, *, category=<class 'DeprecationWarning'>, stacklevel=1)

Mark a callable, class or method as deprecated.

See module docstring for full details. API mirrors warnings.deprecated() from Python 3.13 (PEP 702).

Return type:

Callable[[Union[TypeVar(T, bound= Callable[..., Any]), TypeVar(C, bound= type)]], Union[TypeVar(T, bound= Callable[..., Any]), TypeVar(C, bound= type)]]

gridr.scaling.shmutils.shmarray_wrap(func)

A decorator to automatically load and close SharedMemoryArray instances passed as arguments to a wrapped function.

This helper function simplifies working with SharedMemoryArray objects by automatically handling their load() and close() operations. It’s intended for functions that operate on NumPy arrays but might receive SharedMemoryArray instances as inputs.

Parameters:

func (callable) – The function to be wrapped. Its arguments will be inspected for SharedMemoryArray instances.

Returns:

A wrapper function that handles the loading and closing of SharedMemoryArray arguments before and after executing the original func.

Return type:

callable

Notes

This decorator should be used with caution as it modifies the arguments passed to the wrapped function by replacing SharedMemoryArray instances with their underlying NumPy arrays. It ensures close() is called on all detected SharedMemoryArray instances, even if the wrapped function raises an exception.