The Barlow Twins method tries to minimize redundancy between the individual components of the embedding vectors.

True if the Barlow Twins paper (Zbontar et al., 2021) describes its objective as pushing the cross-correlation matrix between the embeddings of two distorted views towards the identity. Here the off-diagonal (redundancy-reduction) term reduces redundancy between the components of the embedding vectors. Mostly true if that is right but the claim leaves out the invariance (on-diagonal) term, since it states only one part of the objective. Mixed if redundancy reduction is only a minor or secondary part of how the method works. Mostly false if the method mainly works another way and redundancy reduction is only loosely related to it. False if the method does not minimise redundancy between embedding components at all.

Set before the research began.

1. E1 Barlow Twins: Self-Supervised Learning via Redundancy Reduction ar5iv (arXiv HTML rendering of arXiv:2103.03230) Own record

   “Intuitively, the invariance term of the objective, by trying to equate the diagonal elements of the cross-correlation matrix to 1, makes the embedding invariant to the distortions applied. The redundancy reduction term, by trying to equate the off-diagonal elements of the cross-correlation matrix to 0, decorrelates the different vector components of the embedding.”

2. E2 Barlow Twins: Self-Supervised Learning via Redundancy Reduction ar5iv (arXiv HTML rendering of arXiv:2103.03230) Own record

   “Based on this principle, we propose an objective function which tries to make the cross-correlation matrix computed from twin embeddings as close to the identity matrix as possible.”

3. E3 Barlow Twins: Self-Supervised Learning via Redundancy Reduction ar5iv (arXiv HTML rendering of arXiv:2103.03230) Own record

   “First we find that removing the invariance term (on-diagonal term) or the redundancy reduction term (off-diagonal term) of our loss function leads to worse/collapsed solutions, as expected.”

4. E4 Yann LeCun Team's Barlow Twins Method Boosts SSL in Image Representation via Redundancy Reduction | Synced Synced · 15 March 2021

   “The proposed Barlow Twins is an objective function that addresses this, measuring the cross-correlation matrix between the output features of two identical networks fed with distorted versions to make them as close as possible to the identity matrix while minimizing redundancy between concerned vector components.”

The Barlow Twins paper says its objective pushes the cross-correlation matrix of twin embeddings towards the identity (E2). Its off-diagonal redundancy-reduction term decorrelates the components of the embedding vectors (E1), and the ablation shows the method needs this term (E3). An independent report describes the method the same way (E4). The claim leaves out the on-diagonal invariance term, which the paper treats as an equally required part of the objective, so under the criteria the verdict is mostly true.