lucidrains
5578ac472f
address https://github.com/lucidrains/vit-pytorch/issues/292
2023-12-23 08:11:39 -08:00
lucidrains
d446a41243
share an idea that should be tried if it has not been
2023-11-14 16:55:36 -08:00
lucidrains
0ad09c4cbc
allow channels to be customizable for cvt
2023-10-25 14:47:58 -07:00
Phil Wang
92b69321f4
1.6.2
2023-10-24 12:47:38 -07:00
Artem Lukin
fb4ac25174
Fix typo in LayerNorm ( #285 )
...
Co-authored-by: Artem Lukin <artyom.lukin98@gmail.com >
2023-10-24 12:47:21 -07:00
lucidrains
53fe345e85
no longer needed with einops 0.7
2023-10-19 18:16:46 -07:00
Phil Wang
efb94608ea
readme
2023-10-19 09:38:35 -07:00
lucidrains
51310d1d07
add xcit diagram
2023-10-13 09:18:12 -07:00
Phil Wang
1616288e30
add xcit ( #284 )
...
* add xcit
* use Rearrange layers
* give cross correlation transformer a final norm at end
* document
2023-10-13 09:15:13 -07:00
Jason Chou
9e1e824385
Update README.md ( #283 )
...
`patch_size` is size of patches, not number of patches
2023-10-09 11:33:56 -07:00
lucidrains
bbb24e34d4
give a learned bias to and from registers for maxvit + register token variant
2023-10-06 10:40:26 -07:00
lucidrains
df8733d86e
improvise a max vit with register tokens
2023-10-06 10:27:36 -07:00
lucidrains
680d446e46
document in readme later
2023-10-03 09:26:02 -07:00
lucidrains
3fdb8dd352
fix pypi
2023-10-01 08:14:20 -07:00
lucidrains
a36546df23
add simple vit with register tokens example, cite
2023-10-01 08:11:40 -07:00
lucidrains
d830b05f06
address https://github.com/lucidrains/vit-pytorch/issues/279
2023-09-10 09:32:57 -07:00
Phil Wang
8208c859a5
just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point
2023-08-14 09:48:55 -07:00
Phil Wang
4264efd906
1.4.2
2023-08-14 07:59:35 -07:00
Phil Wang
b194359301
add a simple vit with qknorm, since authors seem to be promoting the technique on twitter
2023-08-14 07:58:45 -07:00
lucidrains
950c901b80
fix linear head in simple vit, thanks to @atkos
2023-08-10 14:36:21 -07:00
Phil Wang
3e5d1be6f0
address https://github.com/lucidrains/vit-pytorch/pull/274
2023-08-09 07:53:38 -07:00
Phil Wang
6e2393de95
wrap up NaViT
2023-07-25 10:38:55 -07:00
Phil Wang
32974c33df
one can pass a callback to token_dropout_prob for NaViT that takes in height and width and calculate appropriate dropout rate
2023-07-24 14:52:40 -07:00
Phil Wang
17675e0de4
add constant token dropout for NaViT
2023-07-24 14:14:36 -07:00
Phil Wang
598cffab53
release NaViT
2023-07-24 13:55:54 -07:00
Phil Wang
23820bc54a
begin work on NaViT ( #273 )
...
finish core idea of NaViT
2023-07-24 13:54:02 -07:00
Phil Wang
e9ca1f4d57
1.2.5
2023-07-24 06:43:24 -07:00
roydenwa
d4daf7bd0f
Support SimpleViT as encoder in MAE ( #272 )
...
support simplevit in mae
2023-07-24 06:43:01 -07:00
Phil Wang
9e3fec2398
fix mpp
2023-06-28 08:02:43 -07:00
Phil Wang
ce4bcd08fb
address https://github.com/lucidrains/vit-pytorch/issues/266
2023-05-20 08:24:49 -07:00
Phil Wang
ad4ca19775
enforce latest einops
2023-05-08 09:34:14 -07:00
Phil Wang
e1b08c15b9
fix tests
2023-03-19 10:52:47 -07:00
Phil Wang
c59843d7b8
add a version of simple vit using flash attention
2023-03-18 09:41:39 -07:00
lucidrains
9a8e509b27
separate a simple vit from mp3, so that simple vit can be used after being pretrained
2023-03-07 19:31:10 -08:00
Phil Wang
258dd8c7c6
release mp3, contributed by @Vishu26
2023-03-07 14:29:45 -08:00
Srikumar Sastry
4218556acd
Add Masked Position Prediction ( #260 )
...
* Create mp3.py
* Implementation: Position Prediction as an Effective Pretraining Strategy
* Added description for Masked Position Prediction
* MP3 image added
2023-03-07 14:28:40 -08:00
Phil Wang
f621c2b041
typo
2023-03-04 20:30:02 -08:00
Phil Wang
5699ed7d13
double down on dual patch norm, fix MAE and Simmim to be compatible with dual patchnorm
2023-02-10 10:39:50 -08:00
Phil Wang
46dcaf23d8
seeing a signal with dual patchnorm in another repository, fully incorporate
2023-02-06 09:45:12 -08:00
Phil Wang
bdaf2d1491
adopt dual patchnorm paper for as many vit as applicable, release 1.0.0
2023-02-03 08:11:29 -08:00
Phil Wang
500e23105a
need simple vit with patch dropout for another project
2022-12-05 10:47:36 -08:00
Phil Wang
89e1996c8b
add vit with patch dropout, fully embrace structured dropout as multiple papers are now corroborating each other
2022-12-02 11:28:11 -08:00
Phil Wang
2f87c0cf8f
offer 1d versions, in light of https://arxiv.org/abs/2211.14730
2022-12-01 10:31:05 -08:00
Phil Wang
59c8948c6a
try to fix tests
2022-10-29 11:44:17 -07:00
Phil Wang
cb6d749821
add a 3d version of cct, addressing https://github.com/lucidrains/vit-pytorch/issues/238 0.38.1
2022-10-29 11:35:06 -07:00
Phil Wang
6ec8fdaa6d
make sure global average pool can be used for vivit in place of cls token
2022-10-24 19:59:48 -07:00
Phil Wang
13fabf901e
add vivit
2022-10-24 09:34:04 -07:00
Ryan Russell
c0eb4c0150
Improving Readability ( #220 )
...
Signed-off-by: Ryan Russell <git@ryanrussell.org >
Signed-off-by: Ryan Russell <git@ryanrussell.org >
2022-10-17 10:42:45 -07:00
Phil Wang
5f1a6a05e9
release updated mae where one can more easily visualize reconstructions, thanks to @Vishu26
2022-10-17 10:41:46 -07:00
Srikumar Sastry
9a95e7904e
Update mae.py ( #242 )
...
update mae so decoded tokens can be easily reshaped back to visualize the reconstruction
2022-10-17 10:41:10 -07:00
