Recurrent Layers
SimpleRNN
polyaxon.layers.recurrent.SimpleRNN(mode, num_units, activation='sigmoid', dropout=None, num_layers=1, shared_layers=False, bias=True, weights_init=None, return_seq=False, return_state=False, initial_state=None, dynamic=False, trainable=True, restore=True, name='SimpleRNN')
Simple RNN (Simple Recurrent Layer.)

Output: if
return_seq
: 3D Tensor [samples, timesteps, output dim]. else: 2D Tensor [samples, output dim].

Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  num_units:
int
, number of units for this layer.  activation:
str
(name) orfunction
(returning aTensor
). Default: 'sigmoid'.  dropout:
tuple
offloat
: (1  input_keep_prob, 1  output_keep_prob). The input and output keep probability.  num_layers:
int
how many times to stack the cell.  shared_layers:
bool
To share the same parameters in all created layers.  bias:
bool
. If True, a bias is used.  weights_init:
str
(name) orTensor
. Weights initialization.  return_seq:
bool
. If True, returns the full sequence instead of last sequence output only.  return_state:
bool
. If True, returns a tuple with output and states: (output, states).
 initial_state:
Tensor
. An initial state for the RNN. This must be a tensor of appropriate type and shape [batch_size x cell.state_size].  dynamic:
bool
. If True, dynamic computation is performed. It will not compute RNN steps above the sequence length. Note that because TF requires to feed sequences of same length, 0 is used as a mask. So a sequence padded with 0 at the end must be provided. When computation is performed, it will stop when it meets a step with a value of 0.  trainable:
bool
. If True, weights will be trainable.  restore:
bool
. If True, this layer weights will be restored when loading a model.  name:
str
. A name for this layer (optional).
 mode:
LSTM
polyaxon.layers.recurrent.LSTM(mode, num_units, activation='tanh', inner_activation='sigmoid', dropout=None, num_layers=1, shared_layers=False, bias=True, weights_init=None, forget_bias=1.0, return_seq=False, return_state=False, initial_state=None, dynamic=False, trainable=True, restore=True, name='LSTM')
LSTM (Long Short Term Memory Recurrent Layer).

Output: if
return_seq
: 3D Tensor [samples, timesteps, output dim]. else: 2D Tensor [samples, output dim].

Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  num_units:
int
, number of units for this layer.  activation:
str
(name) orfunction
(returning aTensor
). Default: 'tanh'.  inner_activation:
str
(name) orfunction
(returning aTensor
). LSTM inner activation. Default: 'sigmoid'.  dropout:
tuple
offloat
: (1  input_keep_prob, 1  output_keep_prob). The input and output keep probability.  num_layers:
int
how many times to stack the cell.  shared_layers:
bool
To share the same parameters in all created layers.  bias:
bool
. If True, a bias is used.  weights_init:
str
(name) orTensor
. Weights initialization.  forget_bias:
float
. Bias of the forget gate. Default: 1.0.  return_seq:
bool
. If True, returns the full sequence instead of last sequence output only.  return_state:
bool
. If True, returns a tuple with output and states: (output, states).
 initial_state:
Tensor
. An initial state for the RNN. This must be a tensor of appropriate type and shape [batch_size x cell.state_size].  dynamic:
bool
. If True, dynamic computation is performed. It will not compute RNN steps above the sequence length. Note that because TF requires to feed sequences of same length, 0 is used as a mask. So a sequence padded with 0 at the end must be provided. When computation is performed, it will stop when it meets a step with a value of 0.  trainable:
bool
. If True, weights will be trainable.  restore:
bool
. If True, this layer weights will be restored when loading a model.  name:
str
. A name for this layer (optional).
 mode:

References: Long Short Term Memory, Sepp Hochreiter & Jurgen Schmidhuber, Neural Computation 9(8): 17351780, 1997.

Links:
 [http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf] (http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf)
GRU
polyaxon.layers.recurrent.GRU(mode, num_units, activation='tanh', inner_activation='sigmoid', dropout=None, num_layers=1, shared_layers=False, bias=True, weights_init=None, return_seq=False, return_state=False, initial_state=None, dynamic=False, trainable=True, restore=True, name='GRU')
GRU (Gated Recurrent Unit Layer).

Output: if
return_seq
: 3D Tensor [samples, timesteps, output dim]. else: 2D Tensor [samples, output dim].

Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  num_units:
int
, number of units for this layer.  activation:
str
(name) orfunction
(returning aTensor
). Default: 'tanh'.  inner_activation:
str
(name) orfunction
(returning aTensor
). GRU inner activation. Default: 'sigmoid'.  dropout:
tuple
offloat
: (1  input_keep_prob, 1  output_keep_prob). The input and output keep probability.  num_layers:
int
how many times to stack the cell.  shared_layers:
bool
To share the same parameters in all created layers.  bias:
bool
. If True, a bias is used.  weights_init:
str
(name) orTensor
. Weights initialization.  return_seq:
bool
. If True, returns the full sequence instead of last sequence output only.  return_state:
bool
. If True, returns a tuple with output and states: (output, states).
 initial_state:
Tensor
. An initial state for the RNN. This must be a tensor of appropriate type and shape [batch_size x cell.state_size].  dynamic:
bool
. If True, dynamic computation is performed. It will not compute RNN steps above the sequence length. Note that because TF requires to feed sequences of same length, 0 is used as a mask. So a sequence padded with 0 at the end must be provided. When computation is performed, it will stop when it meets a step with a value of 0.  trainable:
bool
. If True, weights will be trainable.  restore:
bool
. If True, this layer weights will be restored when loading a model.  name:
str
. A name for this layer (optional).
 mode:

References: Learning Phrase Representations using RNN Encoderâ€“Decoder for Statistical Machine Translation, K. Cho et al., 2014.

Links:
BidirectionalRNN
polyaxon.layers.recurrent.BidirectionalRNN(mode, rnncell_fw, rnncell_bw, return_seq=False, return_states=False, initial_state_fw=None, initial_state_bw=None, dynamic=False, name='BiRNN')
Bidirectional RNN.
Build a bidirectional recurrent neural network, it requires 2 RNN Cells to process sequence in forward and backward order. Any RNN Cell can be used i.e. SimpleRNN, LSTM, GRU... with its own parameters. But the two cells number of units must match.

Output: if
return_seq
: 3D Tensor [samples, timesteps, output dim]. else: 2D Tensor Layer [samples, output dim].

Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  rnncell_fw:
RNNCell
. The RNN Cell to use for foward computation.  rnncell_bw:
RNNCell
. The RNN Cell to use for backward computation.  return_seq:
bool
. If True, returns the full sequence instead of last sequence output only.  return_states:
bool
. If True, returns a tuple with output and states: (output, states).
 initial_state_fw:
Tensor
. An initial state for the forward RNN. This must be a tensor of appropriate type and shape [batch_size x cell.state_size].  initial_state_bw:
Tensor
. An initial state for the backward RNN. This must be a tensor of appropriate type and shape [batch_size x cell.state_size].  dynamic:
bool
. If True, dynamic computation is performed. It will not compute RNN steps above the sequence length. Note that because TF requires to feed sequences of same length, 0 is used as a mask. So a sequence padded with 0 at the end must be provided. When computation is performed, it will stop when it meets a step with a value of 0.  name:
str
. A name for this layer (optional).
 mode:
BasicRNNCell
polyaxon.layers.recurrent.BasicRNNCell(mode, num_units, activation='tanh', bias=True, weights_init=None, trainable=True, restore=True, name='BasicRNNCell')
The most basic RNN cell with custom params.
 Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  num_units:
int
, number of units for this layer.  activation:
str
(name) orfunction
(returning aTensor
). Default: 'tanh'.  bias:
bool
. If True, a bias is used.  weights_init:
str
(name) orTensor
. Weights initialization.  trainable:
bool
. If True, weights will be trainable.  restore:
bool
. If True, this layer weights will be restored when loading a model.  name:
str
. A name for this layer (optional).
 mode:
GRUCell
polyaxon.layers.recurrent.GRUCell(mode, num_units, activation='tanh', inner_activation='sigmoid', bias=True, weights_init=None, trainable=True, restore=True, name='GRUCell')
Gated Recurrent Unit cell with custom params.
 Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  num_units:
int
, number of units for this layer.  activation:
str
(name) orfunction
(returning aTensor
). Default: 'tanh'.  inner_activation:
str
(name) orfunction
(returning aTensor
). GRU inner activation. Default: 'sigmoid'.  bias:
bool
. If True, a bias is used.  weights_init:
str
(name) orTensor
. Weights initialization.  trainable:
bool
. If True, weights will be trainable.  restore:
bool
. If True, this layer weights will be restored when loading a model.  name:
str
. A name for this layer (optional).
 mode:
BasicLSTMCell
polyaxon.layers.recurrent.BasicLSTMCell(mode, num_units, forget_bias=1.0, state_is_tuple=True, activation='tanh', inner_activation='sigmoid', bias=True, weights_init=None, batch_norm=False, trainable=True, restore=True, name='BasicLSTMCell')
Basic LSTM recurrent network cell with custo\m params.
The implementation is based on: http://arxiv.org/abs/1409.2329.
We add forget_bias (default: 1) to the biases of the forget gate in order to reduce the scale of forgetting in the beginning of the training.
It does not allow cell clipping, a projection layer, and does not use peephole connections: it is the basic baseline.
For advanced models, please use the full LSTMCell that follows.
 Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  num_units:
int
, number of units for this layer.  forget_bias:
float
. Bias of the forget gate. Default: 1.0.  state_is_tuple: If True, accepted and returned states are ntuples, where
n = len(cells)
. If False, the states are all concatenated along the column axis. This latter behavior will soon be deprecated.  activation:
str
(name) orfunction
(returning aTensor
). Default: 'tanh'.  inner_activation:
str
(name) orfunction
(returning aTensor
). GRU inner activation. Default: 'sigmoid'.  bias:
bool
. If True, a bias is used.  weights_init:
str
(name) orTensor
. Weights initialization.  batch_norm:
bool
. If True, use batch normalization for this cell.  trainable:
bool
. If True, weights will be trainable.  restore:
bool
. If True, this layer weights will be restored when loading a model.  name:
str
. A name for this layer (optional).
 mode:
DropoutWrapper
polyaxon.layers.recurrent.DropoutWrapper(mode, cell, input_keep_prob=1.0, output_keep_prob=1.0, seed=None, name='DropoutWrapper')
Operator adding dropout to inputs and outputs of the given cell.
Creates a cell with added input and/or output dropout.
Dropout is never used on the state.

Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  cell: an RNNCell, a projection to output_size is added to it.
 input_keep_prob: unit Tensor or float between 0 and 1, input keep probability; if it is float and 1, no input dropout will be added.
 output_keep_prob: unit Tensor or float between 0 and 1, output keep probability; if it is float and 1, no output dropout will be added.
 seed: (optional) integer, the randomness seed.
 mode:

Raises:
 TypeError: if cell is not an RNNCell.
 ValueError: if keep_prob is not between 0 and 1.
MultiRNNCell
polyaxon.layers.recurrent.MultiRNNCell(mode, cells, state_is_tuple=True, name='MultiRNNCell')
RNN cell composed sequentially of multiple simple cells.
Create a RNN cell composed sequentially of a number of RNNCells.

Args:
 mode:
str
, Specifies if this training, evaluation or prediction. SeeModes
.  cells: list of RNNCells that will be composed in this order.
 state_is_tuple: If True, accepted and returned states are ntuples, where
n = len(cells)
. If False, the states are all concatenated along the column axis. This latter behavior will soon be deprecated.
 mode:

Raises:
 ValueError: if cells is empty (not allowed), or at least one of the cells
returns a state tuple but the flag
state_is_tuple
isFalse
.
 ValueError: if cells is empty (not allowed), or at least one of the cells
returns a state tuple but the flag