vit-pytorch

__init__.py

no longer needed with einops 0.7

2023-10-19 18:16:46 -07:00

accept_video_wrapper.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

ats_vit.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

cait.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

cct_3d.py

add option to set frame padding for 3D CCT (#339 )

2025-01-04 07:55:27 -08:00

cct.py

Make all CCT regularization parameters user-configurable. (#346 )

2025-08-17 18:07:25 -07:00

cross_vit.py

address https://github.com/lucidrains/vit-pytorch/issues/304

2024-04-17 09:40:03 -07:00

crossformer.py

maxvit intent to build (#211 )

2022-04-06 16:12:17 -07:00

cvt.py

allow channels to be customizable for cvt

2023-10-25 14:47:58 -07:00

deepvit.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

dino.py

implement SOTA new self-supervised learning technique from facebook for vision transformers, Dino

2021-05-02 14:00:36 -07:00

distill.py

fix distill

2025-12-10 15:52:10 -08:00

efficient.py

double down on dual patch norm, fix MAE and Simmim to be compatible with dual patchnorm

2023-02-10 10:39:50 -08:00

es_vit.py

add EsViT, by popular request, an alternative to Dino that is compatible with efficient ViTs with accounting for regional self-supervised loss

2022-05-03 10:29:29 -07:00

extractor.py

offer way for extractor to return latents without detaching them

2022-07-16 16:22:40 -07:00

jumbo_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

learnable_memory_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

lejepa.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

levit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

local_vit.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

look_vit.py

attention re-use in lookup vit should use pre-softmax attention matrix

2024-07-19 19:23:38 -07:00

mae.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

max_vit_with_registers.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

max_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

mobile_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

mp3.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

mpp.py

fix mpp

2023-06-28 08:02:43 -07:00

na_vit_nested_tensor_3d.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

na_vit_nested_tensor.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

na_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

nest.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

normalized_vit.py

fix multiheaded qk rmsnorm in nViT

2024-10-10 19:15:17 -07:00

parallel_vit.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

pit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

recorder.py

return None from extractor if no attention layers

2022-01-28 17:49:58 -08:00

regionvit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

rvt.py

add a hypersphere vit, adapted from https://arxiv.org/abs/2410.01131

2024-10-09 07:32:25 -07:00

scalable_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

sep_vit.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

simmim.py

double down on dual patch norm, fix MAE and Simmim to be compatible with dual patchnorm

2023-02-10 10:39:50 -08:00

simple_flash_attn_vit_3d.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_flash_attn_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_uvit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit_1d.py

address https://github.com/lucidrains/vit-pytorch/pull/274

2023-08-09 07:53:38 -07:00

simple_vit_3d.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit_attn_residual.py

first attention residual should be disabled, cleanup

2026-04-04 17:42:05 -07:00

simple_vit_with_fft.py

address https://github.com/lucidrains/vit-pytorch/issues/292

2023-12-23 08:11:39 -08:00

simple_vit_with_hyper_connections.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit_with_patch_dropout.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit_with_qk_norm.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit_with_register_tokens.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit_with_value_residual.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

simple_vit.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

t2t.py

fix t2t vit having two layernorms, and make final layernorm in distillation wrapper configurable, default to False for vit

2024-06-11 15:12:53 -07:00

twins_svt.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

vaat.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vat_siglip.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vat.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vit_1d.py

Fix typo in LayerNorm (#285 )

2023-10-24 12:47:21 -07:00

vit_3d.py

address https://github.com/lucidrains/vit-pytorch/issues/352

2025-12-02 05:21:52 -08:00

vit_for_small_dataset.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

vit_nd_pope.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vit_nd_rotary.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vit_nd.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vit_with_decorr.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vit_with_keel_post_ln.py

add an attention residual example (kimi team) as well as dino / byol redone with sigreg (lejepa)

2026-03-27 07:51:12 -07:00

vit_with_patch_dropout.py

just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point

2023-08-14 09:48:55 -07:00

vit_with_patch_merger.py

address https://github.com/lucidrains/vit-pytorch/pull/274

2023-08-09 07:53:38 -07:00

vit.py

allow for no final output head on the vit

2026-01-06 13:00:48 -08:00

vivit.py

Fix ViViT Transformer not passing use_flash_attn to Attention and duplicate mask reshape (#360 )

2026-02-11 11:49:31 -08:00

xcit.py

add xcit (#284 )

2023-10-13 09:15:13 -07:00