Soft labelling

dlordinal.soft_labelling.get_beta_soft_labels(J: int, params_set: str | Dict[int, List] = 'standard')

Get soft labels for each of the J classes using a beta distributions and the parameter defined in the params_set as described in Vargas et al.[1].

Parameters:

• J (int) – Number of classes or splits.

• params_set (str or dict[int, list], default="standard") – The set of parameters of the beta distributions employed to generate the soft labels. It can be one of the keys in the _beta_params_sets dictionary. Alternatively, it can be a dictionary with the same structure as the items of the _beta_params_sets dictionary. The keys of the dictionary must be the number of classes and the values must be a list of lists with the parameters of the beta distributions for each class. The list for each class must have three parameters \([p,q,a]\) where \(p\) and \(q\) are the shape parameters of the beta distribution and \(a\) is the scaling parameter. Example: {3: [[1, 4, 1], [4, 4, 1], [4, 1, 1]]} for three classes with the parameters \([1,4,1]\), \([4,4,1]\) and \([4,1,1]\) for each class respectively.

Raises:

ValueError – If J is not a positive integer or if params_set is not a valid key.

Returns:

probs – List of J elements where each elements is also a list of J elements. Each inner list represents the soft label of class j for each of the J classes. For example: probs[j] is the soft label for class j. Then, probs[j][k] is the probability of assigning class k to the instance that belongs to class j.

Return type:

list

Example

>>> from dlordinal.soft_labelling import get_beta_softlabels
>>> get_beta_softlabels(3)
[[0.802469132197531, 0.18518518474074064, 0.01234567906172801],
[0.17329675405578432, 0.6534064918884313, 0.1732967540557846],
[0.012345679061728405, 0.1851851847407408, 0.8024691321975309]]
>>> get_beta_softlabels(5)
[[0.8322278330066323, 0.15097599903815717, 0.01614079995258888,
0.0006528000025611824, 2.5600000609360407e-06], [0.16306230596685573,
0.6740152119811103, 0.15985465217901373, 0.003067150177798128,
6.796952229937148e-07], [0.0005973937258183486, 0.16304604740785777,
0.6727131177326486, 0.16304604740785367, 0.000597393725820794],
[6.796952207410873e-07, 0.0030671501777993896, 0.15985465217901168,
0.6740152119811109, 0.16306230596685678], [2.560000061440001e-06,
0.0006528000025600005, 0.01614079995258879, 0.1509759990381568,
0.8322278330066332]]

dlordinal.soft_labelling.get_binomial_soft_labels(J)

Get soft labels for the binomial distribution for J classes or splits using the approach described in Liu et al.[2]. The \([0,1]\) interval is split into J intervals and the probability for each interval is computed as the difference between the value of the binomial probability function for the interval boundaries. The probability for the first interval is computed as the value of the binomial probability function for the first interval boundary.

The binomial distributions employed are denoted as \(\text{b}(k, n-1, p)\) where \(k\) is given by the order of the class for which the probability is computed, and \(p\) is given by \(0.1 + (0.9-0.1) / (n-1) * j\) where \(j\) is is the order of the target class.

Parameters:

J (int) – Number of classes or splits.

Raises:

ValueError – If J is not a positive integer greater than 1.

Returns:

probs – Matrix of probabilities where each row represents the true class and each column the probability for class j.

Return type:

2d array-like of shape (J, J)

Example

>>> from dlordinal.soft_labelling import get_binomial_soft_labels
>>> get_binomial_soft_labels(5)
array([[6.561e-01, 2.916e-01, 4.860e-02, 3.600e-03, 1.000e-04],
        [2.401e-01, 4.116e-01, 2.646e-01, 7.560e-02, 8.100e-03],
        [6.250e-02, 2.500e-01, 3.750e-01, 2.500e-01, 6.250e-02],
        [8.100e-03, 7.560e-02, 2.646e-01, 4.116e-01, 2.401e-01],
        [1.000e-04, 3.600e-03, 4.860e-02, 2.916e-01, 6.561e-01]])

dlordinal.soft_labelling.get_exponential_soft_labels(J, p=1.0, tau=1.0)

Get soft labels from exponential distribution for J classes or splits as described in Liu et al.[2] and Vargas et al.[3]. The \([0,1]\) interval is split into J intervals and the probability for each interval is computed as the difference between the value of the exponential function for the interval boundaries. The probability for the first interval is computed as the value of the exponential function for the first interval boundary. Then, a softmax function is applied to each row of the resulting matrix to obtain valid probabilities.

The aforementioned exponential function is defined as follows: \(f(x; k, p, \tau) = \frac{-|x - k|^p}{\tau}\) where \(k\) is given by the order of the true class for which the probability is computed, \(p\) is the exponent parameter and \(\tau\) is the scaling parameter.

Parameters:

• J (int) – Number of classes.

• p (float, default=1.0) – Exponent parameter \(p\).

• tau (float, default=1.0) – Scaling parameter \(\tau\).

Raises:

ValueError – If J is not a positive integer greater than 1.

Returns:

probs – Matrix of probabilities where each row represents the true class and each column the probability for class j.

Return type:

2d array-like of shape (J, J)

Example

>>> from dlordinal.soft_labelling import get_exponential_soft_labels
>>> get_exponential_soft_labels(5)
array([[0.63640865, 0.23412166, 0.08612854, 0.03168492, 0.01165623],
[0.19151597, 0.52059439, 0.19151597, 0.07045479, 0.02591887],
[0.06745081, 0.1833503 , 0.49839779, 0.1833503 , 0.06745081],
[0.02591887, 0.07045479, 0.19151597, 0.52059439, 0.19151597],
[0.01165623, 0.03168492, 0.08612854, 0.23412166, 0.63640865]])

dlordinal.soft_labelling.get_general_triangular_params(J: int, alphas: ndarray, verbose: int = 0)

Get the parameters (\(a\), \(b\) and \(c\)) for the general triangular distribution. The parameters are computed using the \(\boldsymbol{\alpha}\) values and the number of classes \(J\). The \(\boldsymbol{\alpha}\) vector contains two \(\alpha\) values for each class or split. \(\alpha_0\) should always be 0, given that the error on the left side of the first class is always 0. In the same way, the \(\alpha_{2J}\) value should always be 0, given that the error on the right side of the last class is always 0. The \(\alpha\) values for the other classes should be between 0 and 1.

The parameters \(a\), \(b\) and \(c\) for class \(j\) are computed as described in Vargas et al.[4].

Parameters:

• J (int) – Number of classes or splits.

• alphas (np.ndarray) – Array that represents the error on the left and right side of each class. It is the \(\boldsymbol{\alpha}\) vector described in Vargas et al.[4].

• verbose (int, optional) – Verbosity level, by default 0.

Raises:

ValueError – If the number of classes \(J\) is less than 2. If the \(\boldsymbol{\alpha}\) vector is not a numpy array of shape \((2J,)\).

Returns:

List of dictionaries with the parameters for each class.

Return type:

list

Example

>>> from dlordinal.soft_labelling import get_general_triangular_params
>>> get_general_triangular_params(5, [0, 0.1, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.1, 0])
[{'alpha2j_1': 0, 'alpha2j': 0.05, 'a': 0, 'b': 0.25760143110525874, 'c': 0}, {'alpha2j_1': 0.05, 'alpha2j': 0.05, 'a': 0.153752470442574, 'b': 0.446247529557426, 'c': 0.3}, {'alpha2j_1': 0.05, 'alpha2j': 0.05, 'a': 0.353752470442574, 'b': 0.646247529557426, 'c': 0.5}, {'alpha2j_1': 0.05, 'alpha2j': 0.1, 'a': 0.550779686438060, 'b': 0.875486049708105, 'c': 0.7}, {'alpha2j_1': 0.0, 'alpha2j': 0, 'a': 0.8, 'b': 1, 'c': 1}]

dlordinal.soft_labelling.get_general_triangular_soft_labels(J: int, alphas: ndarray, verbose: int = 0)

Get soft labels using triangular distributions for J classes or splits. The \([0,1]\) interval is split into J intervals and the probability for each interval is computed as the difference between the value of the triangular distribution function for the interval boundaries. The probability for the first interval is computed as the value of the triangular distribution function for the first interval boundary.

The triangular distribution function is denoted as \(\text{f}(x, a, b, c)\). The parameters \(a\), \(b\) and \(c\) for class \(j\) are computed as described in Vargas et al.[4].

Parameters:

• J (int) – Number of classes.

• alphas (np.ndarray) – Array of alphas.

• verbose (int, optional) – Verbosity level, by default 0.

Raises:

ValueError – If J is not a positive integer greater than 1. If alphas is not a numpy array of shape \((2J,)\).

Returns:

probs – Matrix of probabilities where each row represents the true class and each column the probability for class j.

Return type:

2d array-like of shape (J, J)

Example

>>> from dlordinal.soft_labelling import get_general_triangular_soft_labels
>>> get_general_triangular_soft_labels(
...     5,
...     [0, 0.1, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.1, 0]
... )
array([[0.95, 0.05, 0.  , 0.  , 0.  ],
       [0.05, 0.9 , 0.05, 0.  , 0.  ],
       [0.  , 0.05, 0.9 , 0.05, 0.  ],
       [0.  , 0.  , 0.05, 0.85, 0.1 ],
       [0.  , 0.  , 0.  , 0.  , 1.  ]], dtype=float32)

dlordinal.soft_labelling.get_geometric_soft_labels(J: int, alphas: float | list = 0.1)

Get soft labels based on the discrete geometric distribution according to Haas and Hüllermeier[5].

Parameters:

• J (int) – Number of classes.

• alphas (float or list, default=0.1) –

  The smoothing factor(s) for geometric distribution-based unimodal smoothing.

  • Single alpha value: When a single alpha value in the range [0, 1], e.g., 0.1, is provided, all classes will be smoothed equally and symmetrically. This is done by deducting alpha from the actual class, \(1 - \alpha\), and allocating \(\alpha\) to the rest of the classes, decreasing monotonically from the actual class in the form of the geometric distribution.

    Formula ( with \(j\) as the index of the observed class in the one-hot encoded label and \(k\) the current class):

    \[\begin{split}p_{i}^G(k) = \begin{cases} 1-\alpha & \text{if } k = j \\ 1/G_{i} \; \alpha^{|j-k|+1}(1-\alpha) & \text{if } k \neq j \\ \end{cases}.\end{split}\]

    Normalizing constant:

    \[G_{i} = p_{i}^G(k \neq j) = \sum_{k \neq j} \alpha^{|j-k|}(1-\alpha).\]

  • List of alpha values: Alternatively, a list of size num_classes can be provided to specify class-wise symmetric smoothing factors. An example for five classes is: [0.2, 0.05, 0.1, 0.15, 0.1].

  • List of smoothing relations: To control the fraction of the left-over probability mass \(\alpha\) allocated to the left (\(F_l \in [0,1]\)) and right (\(F_r \in [0,1]\)) sides of the true class, with \(F_l + F_r = 1\), a list of smoothing relations of the form \((\alpha, F_l, F_r)\) can be specified. This enables asymmetric unimodal smoothing. An example for five classes is: [(0.2, 0.0, 1.0), (0.05, 0.8, 0.2), (0.1, 0.5, 0.5), (0.15, 0.6, 0.4), (0.1, 1.0, 0.0)].

    \[\begin{split}p_{i}^G(k) = \begin{cases} 1-\alpha_{j} & \text{if } k = j \\ 1/G_{i} \; F_{l,j} \; \alpha_{j}^{(j-k)+1}(1-\alpha_{j}) & \text{if } k < j \\ 1/G_{i} \; F_{r,j} \; \alpha_{j}^{(k-j)+1}(1-\alpha_{j}) & \text{if } k > j \\ \end{cases}\end{split}\]

Raises:

ValueError – If J is not a positive integer greater than 1. If smoothing values in alphas are not in [0,1]. If alphas is a list and size of alphas is not equal to J. If alphas is not a float, list of floats, or list of tuples. If probability fractions \(F_l \in [0,1]\) and \(F_r \in [0,1]\) do not sum to one.

Returns:

probs – Matrix of probabilities where each row represents the true class and each column the probability for class j.

Return type:

2d array-like of shape (J, J)

Example

>>> from dlordinal.soft_labelling import get_geometric_soft_labels
>>> get_geometric_soft_labels(5)
array([[0.9       , 0.090009  , 0.0090009 , 0.00090009, 0.00009001],
   [0.04739336, 0.9       , 0.04739336, 0.00473934, 0.00047393],
   [0.00454545, 0.04545455, 0.9       , 0.04545455, 0.00454545],
   [0.00047393, 0.00473934, 0.04739336, 0.9       , 0.04739336],
   [0.00009001, 0.00090009, 0.0090009 , 0.090009  , 0.9       ]])
>>> get_geometric_soft_labels(5, alphas=0.3)
array([[0.7       , 0.21171489, 0.06351447, 0.01905434, 0.0057163 ],
   [0.12552301, 0.7       , 0.12552301, 0.0376569 , 0.01129707],
   [0.03461538, 0.11538462, 0.7       , 0.11538462, 0.03461538],
   [0.01129707, 0.0376569 , 0.12552301, 0.7       , 0.12552301],
   [0.0057163 , 0.01905434, 0.06351447, 0.21171489, 0.7       ]])
>>> get_geometric_soft_labels(5, alphas=[0.3,0.2,0.05,0.02,0.5])
array([[0.7       , 0.21171489, 0.06351447, 0.01905434, 0.0057163 ],
   [0.08928571, 0.8       , 0.08928571, 0.01785714, 0.00357143],
   [0.00119048, 0.02380952, 0.95      , 0.02380952, 0.00119048],
   [0.00000396, 0.00019798, 0.00989903, 0.98      , 0.00989903],
   [0.03333333, 0.06666667, 0.13333333, 0.26666667, 0.5       ]])
>>> get_geometric_soft_labels(5, alphas=[(0.2, 0.0, 1.0), (0.05, 0.8, 0.2), (0.1, 0.5, 0.5), (0.15, 0.6, 0.4), (0.1, 1.0, 0.0)])
array([[0.8       , 0.16025641, 0.03205128, 0.00641026, 0.00128205],
   [0.04      , 0.95      , 0.00950119, 0.00047506, 0.00002375],
   [0.00454545, 0.04545455, 0.9       , 0.04545455, 0.00454545],
   [0.00172708, 0.01151386, 0.07675906, 0.85      , 0.06      ],
   [0.00009001, 0.00090009, 0.0090009 , 0.090009  , 0.9       ]])

dlordinal.soft_labelling.get_poisson_soft_labels(J)

Get soft labels using poisson distributions for J classes or splits using the methodology described in Liu et al.[2]. The \([0,1]\) interval is split into J intervals and the probability for each interval is computed as the difference between the value of the poisson probability function for the interval boundaries. The probability for the first interval is computed as the value of the poisson probability function for the first interval boundary. Then, a softmax function is applied to each row of the resulting matrix to obtain valid probabilities.

The poisson probability function is denoted as \(\text{p}(k, \lambda)\) where \(k\) is given by the order of the class for which the probability is computed, and \(\lambda\) is given by \(k\) where \(k\) is the order of the target class.

Parameters:

J (int) – Number of classes or splits.

Raises:

ValueError – If J is not a positive integer greater than 1.

Returns:

probs – Matrix of probabilities where each row represents the true class and each column the probability for class j.

Return type:

2d array-like of shape (J, J)

Example

>>> from dlordinal.soft_labelling import get_poisson_soft_labels
>>> get_poisson_soft_labels(5)
array([[0.23414552, 0.23414552, 0.19480578, 0.17232403, 0.16457916],
    [0.18896888, 0.21635436, 0.21635436, 0.19768881, 0.18063359],
    [0.17822335, 0.19688341, 0.21214973, 0.21214973, 0.20059378],
    [0.17919028, 0.18931175, 0.20370191, 0.21389803, 0.21389803],
    [0.18400408, 0.18903075, 0.19882883, 0.21031236, 0.21782399]])

dlordinal.soft_labelling.get_triangular_soft_labels(J: int, alpha2: float = 0.01, verbose: int = 0)

Get soft labels using triangular distributions for J classes or splits using the approach described in Vargas et al.[6]. The \([0,1]\) interval is split into J intervals and the probability for each interval is computed as the difference between the value of the triangular distribution function for the interval boundaries. The probability for the first interval is computed as the value of the triangular distribution function for the first interval boundary.

The triangular distribution function is denoted as \(\text{p}(x, a, b, c)\) where \(a\), \(b\) and \(c\) are the parameters of the distribution, and are determined by the number of classes \(J\) and the value of the \(\alpha_2\) parameter. The value of \(\alpha_2\) represents the probability that is assigned to the adjacent classes of the target class. The parameters \(a\), \(b\) and \(c\) for class \(j\) are computed as follows:

\[\begin{split}a_j = \begin{cases} 0 & \text{if } j = 1 \\ \frac{2j - 2 - 4j\alpha_2 + 2\alpha_2 \pm \sqrt{2\alpha_2}}{2n(1 - 2\alpha_2)} & \text{if } 1 < j < J\\ 1 + \frac{1}{\pm J (\sqrt{\alpha_3} - 1)} & \text{if } j = J \end{cases}\end{split}\]

\[\begin{split}b_j = \begin{cases} \frac{1}{(1 - \sqrt{\alpha_1})n} & \text{if } j = 1 \\ \frac{2j - 4j\alpha_2 + 2\alpha_2 \pm \sqrt{2\alpha_2}}{2n(1 - 2\alpha_2)} & \text{if } 1 < j < J\\ 1 & \text{if } j = J \end{cases}\end{split}\]

\[\begin{split}c_j = \begin{cases} 0 & \text{if } j = 1 \\ \frac{a + b}{2} & \text{if } 1 < j < J\\ 1 & \text{if } j = J \end{cases}\end{split}\]

The value of \(\alpha_1\), that represents the error for the first class, is computed as follows:

\[\alpha_1 = \left(\frac{1 - \sqrt{1 - 4(1 - 2\alpha_2)(2\alpha_2 - \sqrt{2\alpha_2})}}{2}\right)^2\]

The value of \(\alpha_3\), that represents the error for the last class, is computed as follows:

\[\alpha_3 = \left(\frac{1 - \sqrt{1 - 4\left(\frac{J - 1}{J}\right)^2(1 - 2\alpha_2)(\sqrt{2\alpha_2} (-1 + \sqrt{2\alpha_2}))}}{2}\right)^2\]

The value of \(\alpha_2\) is given by the user.

Parameters:

• J (int) – Number of classes or splits (\(J\)).

• alpha2 (float, optional, default=0.01) – Value of the \(\alpha_2\) parameter.

• verbose (int, optional, default=0) – Verbosity level.

Raises:

ValueError – If J is not a positive integer greater than 1. If alpha2 is not a float between 0 and 1.

Returns:

probs – Matrix of probabilities where each row represents the true class and each column the probability for class j.

Return type:

2d array-like of shape (J, J)

Example

>>> from dlordinal.soft_labelling import get_triangular_soft_labels
>>> get_triangular_soft_labels(5)
array([[0.98845494, 0.01154505, 0.        , 0.        , 0.        ],
       [0.01      , 0.98      , 0.01      , 0.        , 0.        ],
       [0.        , 0.01      , 0.98      , 0.01      , 0.        ],
       [0.        , 0.        , 0.01      , 0.98      , 0.01      ],
       [0.        , 0.        , 0.        , 0.00505524, 0.99494475]])

[1]

Víctor Manuel Vargas, Pedro Antonio Gutiérrez, and César Hervás-Martínez. Unimodal regularisation based on beta distribution for deep ordinal regression. Pattern Recognition, 122:108310, 2022. doi:https://doi.org/10.1016/j.patcog.2021.108310.

[2] (1,2,3)

Xiaofeng Liu, Fangfang Fan, Lingsheng Kong, Zhihui Diao, Wanqing Xie, Jun Lu, and Jane You. Unimodal regularized neuron stick-breaking for ordinal classification. Neurocomputing, 388(7):34–44, 2020. doi:https://doi.org/10.1016/j.neucom.2020.01.025.

[3]

Víctor Manuel Vargas, Pedro Antonio Gutiérrez, Riccardo Rosati, Luca Romeo, Emanuele Frontoni, and César Hervás-Martínez. Exponential loss regularisation for encouraging ordinal constraint to shotgun stocks quality assessment. Applied Soft Computing, 138:110191, 2023. doi:https://doi.org/10.1016/j.asoc.2023.110191.

[4] (1,2,3)

Víctor Manuel Vargas, Antonio Manuel Durán-Rosal, David Guijo-Rubio, Pedro Antonio Gutiérrez, and César Hervás-Martínez. Generalised triangular distributions for ordinal deep learning: novel proposal and optimisation. Information Sciences, 648:119606, 2023. doi:https://doi.org/10.1016/j.ins.2023.119606.

[5]

Stefan Haas and Eyke Hüllermeier. Rectifying bias in ordinal observational data using unimodal label smoothing. In Gianmarco De Francisci Morales, Claudia Perlich, Natali Ruchansky, Nicolas Kourtellis, Elena Baralis, and Francesco Bonchi, editors, Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track - European Conference, ECML PKDD 2023, Turin, Italy, September 18-22, 2023, Proceedings, Part VI, volume 14174 of Lecture Notes in Computer Science, 3–18. Springer, 2023. URL: https://doi.org/10.1007/978-3-031-43427-3_1, doi:10.1007/978-3-031-43427-3_1.

[6]

Víctor Manuel Vargas, Pedro Antonio Gutiérrez, Javier Barbero-Gómez, and César Hervás-Martínez. Soft labelling based on triangular distributions for ordinal classification. Information Fusion, 93:258–267, 2023. doi:https://doi.org/10.1016/j.inffus.2023.01.003.