PMSPCell

Bases: tf.keras.layers.Layer

RNN cell for PMSP model.

See Plaut, McClelland, Seidenberg and Patterson (1996), simulation 3.

Parameters:

Name	Type	Description	Default
tau	float	Time-averaging parameter, from 0 to 1.	required
h_units	int	Number of units in the hidden layer.	required
p_units	int	Number of units in the phonological layer.	required
c_units	int	Number of units in the cleanup layer.	required
h_noise	float	Gaussian noise parameter (in stddev) for hidden layer.	0.0
p_noise	float	Gaussian noise parameter (in stddev) for phonological layer.	0.0
c_noise	float	Gaussian noise parameter (in stddev) for cleanup layer.	0.0
connections	List[str]	List of connections to use, each connection consists of two letters (from, to). Default is ["oh", "ph", "hp", "pp", "cp", "pc"].	None
zero_out_rates	Dict[str, float]	Dictionary of zero-out rates for each connection. Default is {c: 0.0 for c in self.connections}. See PMSP for more details.	None
l2	float	L2 regularization parameter, apply to all trainable weights and biases.	0.0

all_layers_names: List[str] property

List of all layer full names.

build(input_shape)

Build the layer.

call(last_o, last_h, last_p, last_c, training=False, return_internals=False)

Forward pass.

Parameters:

Name	Type	Description	Default
last_o	tf.Tensor	Orthographic inputs from previous timestep, shape: (batch_size, input_units).	required
last_h	tf.Tensor	Hidden layer activations from previous timestep, shape: (batch_size, h_units).	required
last_p	tf.Tensor	Phonology layer activations from previous timestep, shape: (batch_size, p_units).	required
last_c	tf.Tensor	Cleanup layer activations from previous timestep, shape: (batch_size, c_units).	required
training	bool	Whether in training mode.	False
return_internals	bool	Whether to return intermediate inputs to each connection.	False

Returns:

Name	Type	Description
outputs	Dict[str, tf.Tensor]	Activations in each layer {"phonology": ...}, if return_internals=True, also return intermediate inputs in each connection. E.g., {"oh": last_o @ w_{oh}, ...}

get_connection_units(connection)

Get number of output units for a given connection.

Parameters:

Name	Type	Description	Default
connection	str	Connection string, select from self.connections.	required

Returns:

Name	Type	Description
units	int	Number of output units.

get_connections(layer)

Get connections that end with a given layer.

Parameters:

Name	Type	Description	Default
layer	str	Layer name, select from self.all_layers_names.	required

Returns:

Name	Type	Description
connections	List[str]	List of connections that end with the given layer.

Example

if layer = "hidden", return all connections that ends with "h", e.g.: ["oh", "ph"]

cell.get_connections("hidden")
>>> ["oh", "ph"]

reset_states()

Reset all time-averaging states.

zero_out_weights()

Assign zero values to weights by its masks in all connections.

See ZeroOutDense for more details.