Nonlinear Dimensionality Reduction Techniques by Sylvain Lespinats & Benoit Colange & Denys Dutykh

Author:Sylvain Lespinats & Benoit Colange & Denys Dutykh
Language: eng
Format: epub
ISBN: 9783030810269
Publisher: Springer International Publishing

5.3.1 General Principle: SNE

In order to assess the preservation of the neighbourhood structure with respect to the mapping, neighbourhood embedding algorithms measure for each point i, the belonging of any other point j to its soft neighbourhood in both spaces. This gives the neighbourhood membership degrees in the data space and in the embedding space. In the original method SNE [87], these membership degrees are defined using Gaussian kernels as:

(5.13)

where {σ i} and {s i} are scale parameters in the data and embedding spaces. In the neighbourhood retrieval perspective, these normalized neighbourhood membership degree are sometimes interpreted as a probability of selecting a neighbour j in the neighbourhood of a given point i [189].

This softmin formulation benefits from the shift-invariance property [107], allowing to mitigate the phenomenon of norm concentration (see Sect. 2.​1). This property may partly justify the better suitability of neighbourhood embedding methods for very high dimensional data (compared with MDS).

The neighbourhood membership degrees being normalized (i.e. summing to one), they behave as discrete probability distributions. Hence, they may be compared between the two spaces using the Kullback–Leibler divergence , leading to a point-wise stress:

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.