Activations

built_activation

built_activation(fct, name, collect)

Builds the metric function.

  • Args:
    • fct: the activation function to build.
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

linear

linear(name='Linear', collect=False)

Computes linear/identity function.

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

tanh

tanh(name=None, collect=False)

Computes hyperbolic tangent of x element-wise.

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

sigmoid

sigmoid(name=None, collect=False)

Computes sigmoid of x element-wise: y = 1 / (1 + exp(-x)).

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

softmax

softmax(name=None, collect=False)

Computes softmax activations.

For each batch i and class j we have softmax[i, j] = exp(logits[i, j]) / sum(exp(logits[i]))

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

softplus

softplus(name=None, collect=False)

Computes softplus. log(exp(features) + 1).

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

softsign

softsign(name=None, collect=False)

Computes softsign: features / (abs(features) + 1).

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

relu

relu(name=None, collect=False)

Computes ReLU, rectified linear: max(features, 0).

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

relu6

relu6(name=None, collect=False)

Computes Rectified Linear 6: min(max(features, 0), 6).

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

leaky_relu

leaky_relu(alpha=0.1, name='LeakyReLU', collect=False)

Modified version of ReLU, introducing a nonzero gradient for negative input.

  • Args:
    • alpha: int, the multiplier.
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

prelu

prelu(channel_shared=False, weights_init='zeros', restore=True, name='PReLU', collect=False)

Parametric Rectified Linear Unit.

  • Args:
    • channel_shared:
    • weights_init:
    • restore:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

elu

elu(name=None, collect=False)

Computes Exponential Linear Unit.

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.

crelu

crelu(name='CRelu', collect=False)

Computes Concatenated ReLU.

  • Args:
    • name: operation name.
    • collect: whether to collect this metric under the metric collection.