TimeAveragedRNNCell

Bases: tf.keras.layers.Layer

A simple RNN cell with time-averaging output. It defines one step of compute in an RNN.

Note

This layer serves as an example of how to use MultiInputTimeAveraging in a custom RNN cell.

Parameters:

Name	Type	Description	Default
tau	float	Time-averaging parameter, from 0 to 1.	required
units	int	Number of units in the RNN cell.	required

Forward pass

$$h_t = \tau \cdot \sigma(h_{t-1} w_{hh} + x_t w_{xh} + b) + (1 - \tau) \cdot h_{t-1}$$

Warning

"$\cdot$" is element-wise multiplication.

• $h_t$ is the output (also equal to hidden state) at time $t$.
• $x_t$ is the input at time $t$.
• $\sigma$ is the sigmoid activation function.
• $w_{xh}$ is the weight matrix from input to hidden state.
• $w_{hh}$ is the weight matrix from hidden state to hidden state.
• $b$ is the bias vector.
• $\tau$: time constant, smaller means slower temporal dynamics.

build(input_shape)

Build the underlying layers/weights.

call(inputs, states=None)

Forward pass.

Parameters:

Name	Type	Description	Default
inputs	tf.Tensor	Input tensor $x_t$ with shape (batch_size, input_dim).	required
states	tf.Tensor	Hidden state tensor $h_{t-1}$ with shape (batch_size, units).	None

Returns:

Name	Type	Description
states	tf.Tensor	Hidden state tensor $h_t$.
outputs	tf.Tensor	Output tensor $h_t$. This is the same as the states.

reset_states()

Resetting the time-averaging state to None

Note

This time-averaging state $h_{t-1}$ is handled inside the MultiInputTimeAveraging layer, which taking care of the 2nd part of the forward pass: $(1 - \tau) \cdot h_{t-1}$.

Even though the first part of the forward pass: $\tau \cdot \sigma(h_{t-1} w_{hh} + x_t w_{xh} + b)$ is using the same $h_{t-1}$ values, it is handled explicitly in the call method, so we don't need to reset it here.