Penalty

Description

Info

Parent class: Module

Derived classes: -

This module implements the L1 (Lasso) or L2 (Ridge) regularization function.

It is used to reduce the likelihood of overfitting the model.

Additional sources

Initializing

def __init__(self, mode="l1", weight=1e-2, name=None):

Parameters

Parameter Allowed types Description Default
mode str Regularization type, can be "l1"or "l2" "l1"
weight float Regularization parameter value 1e-2
name str Layer name None

Explanations

-

Examples

import numpy as np
from PuzzleLib.Backend import gpuarray
from PuzzleLib.Modules import Penalty
data = gpuarray.to_gpu(np.random.randn(10, 50).astype(np.float32))

penalty = Penalty()
penalty(data)

grad = gpuarray.to_gpu(np.random.randn(10, 50).astype(np.float32))
penalty.backward(grad)