Core

class streamer.core.buffer.MemBuffer(buffer_size=20, dist_mode='similarity')

Buffer class that maintains size of inputs to the temporal encoding model. The buffer replaces low prediction error inputs with high prediction error values while maintaining the order they were received in.

Parameters:

• buffer_size (int) – The maximum buffer size to maintain

• dist_mode (str) – The distance mode for error calculation (e.g., ‘similarity’, ‘distance’)

add_input(x, err)

Adds a single input and its corresponding error value to the buffer. If the buffer is full, this function takes care of replacing the low prediction error value if the new value is higher.

Parameters:

• x (torch.Tensor) – The input tensor to be added to the buffer

• err (torch.Tensor) – The corresponding error or similarity value.

get_inputs()

Returns the current inputs in the buffer in the correct order they were received.

Returns:

(List(torch.Tensor)): A list of tensors in the buffer

reset_buffer()

Resets the buffer. Can be called after a boundary has been detected

---

class streamer.core.demarcation.EventDemarcation(dem_mode, dist_mode, **kwargs)

Define different Demarcation strategies

Parameters:

• dem_mode (str) – the type of demarcation loss: [‘fixed’, ‘accum’, ‘average’]

• dist_mode (str) – the type of demarcation loss: [‘similarity’, ‘distance’]

• threshold (float) – error or similarity threshold value. Only used with ‘accum’ and ‘fixed’ demarcation modes

• window_size (int) – Window size for moving average. Only used for ‘average’ demarcation mode

• modifier_type (str) – type of change to average: [‘add’, ‘multiply’]. Only used for ‘average’ demarcation mode

• modifier (float) – multiplier to change the average value as threshold. Only used for ‘average’ demarcation mode

__call__(value)

Detect boundary according to demarcation mode and distance mode

Parameters:

value (float) – The new error value

Returns:

(bool): True if the new value in a boundary

reset()

Reset the window for demarcation. Only affects ‘accum’ and ‘average’ demarcation modes

---

class streamer.core.loss.StreamerLoss(dist_mode)

Streamer loss function is applied to minimize the distance between the prediction and actual input. Implemented for Cosine Similarity and Euclidean distance. Use only Cosine Similarity as it backpropagates bounded loss values across layers.

Parameters:

dist_mode (str) – The distance mode: [‘similarity’, ‘distance’]

forward(context, groundtruth)

Calculates the loss value from context and groundtruth

Parameters:

• context (torch.Tensor) – The prediction tensor. shape=[1,3,H,W] or [1,feature_dim]

• groundtruth (torch.Tensor) – The groundtruth tensor. shape=[1,3,H,W] or [1,feature_dim]

Returns:

• (dict): dictionary of all the losses

• (float): the loss value to be used for demarcation (i.e., input to the __call__())

summarize_loss(loss_dict)

Converts losses_dict from the output of forward() to loss value respecting the distance mode. Called by the optimizer

Parameters:

losses_dict (dict) – The full losses dictionary

Returns:

(float): The loss value to be minimized