From efb94608eac729804ebc09b5b02be1fc75c2c6f3 Mon Sep 17 00:00:00 2001 From: Phil Wang Date: Thu, 19 Oct 2023 09:38:35 -0700 Subject: [PATCH] readme --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 89e7a9c..d88d10a 100644 --- a/README.md +++ b/README.md @@ -777,7 +777,7 @@ pred = mbvit_xs(img) # (1, 1000) -This paper introduces the cross correlation attention (abbreviated XCA). One can think of it as doing attention across the features dimension rather than the spatial one (another perspective would be a dynamic 1x1 convolution, the kernel being attention map defined by spatial correlations). +This paper introduces the cross covariance attention (abbreviated XCA). One can think of it as doing attention across the features dimension rather than the spatial one (another perspective would be a dynamic 1x1 convolution, the kernel being attention map defined by spatial correlations). Technically, this amounts to simply transposing the query, key, values before executing cosine similarity attention with learned temperature.