ZF/vit-pytorch
1
0
Fork 0
mirror of https://github.com/lucidrains/vit-pytorch.git synced 2025-12-30 08:02:29 +00:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • fb37832f66 document lucidrains 2023-10-13 09:15:00 -07:00
  • d9679d3e26 give cross correlation transformer a final norm at end lucidrains 2023-10-12 19:51:07 -07:00
  • bcfb0f054a use Rearrange layers lucidrains 2023-10-12 19:48:31 -07:00
  • 3dfb1579f7 add xcit lucidrains 2023-10-12 19:32:50 -07:00
  • 9e1e824385 Update README.md (#283) Jason Chou 2023-10-09 11:33:56 -07:00
  • bbb24e34d4 give a learned bias to and from registers for maxvit + register token variant 1.5.3 lucidrains 2023-10-06 10:40:26 -07:00
  • df8733d86e improvise a max vit with register tokens lucidrains 2023-10-06 10:27:36 -07:00
  • 014df1e6e4 improvise a max vit with register tokens 1.5.2 lucidrains 2023-10-06 10:24:35 -07:00
  • ca7d7e39e3 improvise a max vit with register tokens 1.5.1 lucidrains 2023-10-06 10:22:55 -07:00
  • 680d446e46 document in readme later lucidrains 2023-10-03 09:26:02 -07:00
  • 3fdb8dd352 fix pypi 1.5.0a lucidrains 2023-10-01 08:14:20 -07:00
  • a36546df23 add simple vit with register tokens example, cite 1.5.0 lucidrains 2023-10-01 08:11:40 -07:00
  • d830b05f06 address https://github.com/lucidrains/vit-pytorch/issues/279 1.4.5 lucidrains 2023-09-10 09:32:57 -07:00
  • 8208c859a5 just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point 1.4.4 Phil Wang 2023-08-14 09:48:55 -07:00
  • ad7d0df22c just remove PreNorm wrapper from all ViTs, as it is unlikely to change at this point 1.4.3 Phil Wang 2023-08-14 09:41:40 -07:00
  • 4264efd906 1.4.2 1.4.2 Phil Wang 2023-08-14 07:59:35 -07:00
  • b194359301 add a simple vit with qknorm, since authors seem to be promoting the technique on twitter Phil Wang 2023-08-14 07:58:38 -07:00
  • 950c901b80 fix linear head in simple vit, thanks to @atkos 1.4.1 lucidrains 2023-08-10 14:36:21 -07:00
  • 3e5d1be6f0 address https://github.com/lucidrains/vit-pytorch/pull/274 1.4.0 Phil Wang 2023-08-09 07:53:38 -07:00
  • 6e2393de95 wrap up NaViT 1.2.9 Phil Wang 2023-07-25 10:38:55 -07:00
  • 32974c33df one can pass a callback to token_dropout_prob for NaViT that takes in height and width and calculate appropriate dropout rate Phil Wang 2023-07-24 14:52:40 -07:00
  • cd210905d9 one can pass a callback to token_dropout_prob for NaViT that takes in height and width and calculate appropriate dropout rate 1.2.8 Phil Wang 2023-07-24 14:30:30 -07:00
  • 17675e0de4 add constant token dropout for NaViT 1.2.7 Phil Wang 2023-07-24 14:14:36 -07:00
  • 598cffab53 release NaViT Phil Wang 2023-07-24 13:55:54 -07:00
  • 57a3862b7b release NaViT 1.2.6 Phil Wang 2023-07-24 13:54:31 -07:00
  • 23820bc54a begin work on NaViT (#273) Phil Wang 2023-07-24 13:54:02 -07:00
  • e9ca1f4d57 1.2.5 1.2.5 Phil Wang 2023-07-24 06:43:24 -07:00
  • d4daf7bd0f Support SimpleViT as encoder in MAE (#272) roydenwa 2023-07-24 15:43:01 +02:00
  • 9e3fec2398 fix mpp 1.2.4 Phil Wang 2023-06-28 08:02:43 -07:00
  • ce4bcd08fb address https://github.com/lucidrains/vit-pytorch/issues/266 1.2.2 Phil Wang 2023-05-20 08:24:49 -07:00
  • ad4ca19775 enforce latest einops 1.2.1 Phil Wang 2023-05-08 09:34:14 -07:00
  • e1b08c15b9 fix tests Phil Wang 2023-03-19 10:52:47 -07:00
  • c59843d7b8 add a version of simple vit using flash attention 1.2.0 Phil Wang 2023-03-18 09:41:12 -07:00
  • 9a8e509b27 separate a simple vit from mp3, so that simple vit can be used after being pretrained lucidrains 2023-03-07 19:31:10 -08:00
  • 0ed18c89ea separate a simple vit from mp3, so that simple vit can be used after being pretrained 1.1.1 lucidrains 2023-03-07 19:29:50 -08:00
  • 258dd8c7c6 release mp3, contributed by @Vishu26 1.1.0 Phil Wang 2023-03-07 14:29:45 -08:00
  • 4218556acd Add Masked Position Prediction (#260) Srikumar Sastry 2023-03-07 16:28:40 -06:00
  • f621c2b041 typo Phil Wang 2023-03-04 20:30:02 -08:00
  • 5699ed7d13 double down on dual patch norm, fix MAE and Simmim to be compatible with dual patchnorm 1.0.2 Phil Wang 2023-02-10 10:39:50 -08:00
  • 46dcaf23d8 seeing a signal with dual patchnorm in another repository, fully incorporate 1.0.1 Phil Wang 2023-02-06 09:45:12 -08:00
  • bdaf2d1491 adopt dual patchnorm paper for as many vit as applicable, release 1.0.0 1.0.0 Phil Wang 2023-02-03 08:11:29 -08:00
  • 500e23105a need simple vit with patch dropout for another project Phil Wang 2022-12-05 10:47:36 -08:00
  • 105e97f240 need simple vit with patch dropout for another project 0.40.2 Phil Wang 2022-12-05 10:47:02 -08:00
  • 89e1996c8b add vit with patch dropout, fully embrace structured dropout as multiple papers are now corroborating each other Phil Wang 2022-12-02 11:28:11 -08:00
  • e05152210e add vit with patch dropout, fully embrace structured dropout as multiple papers are now corroborating each other 0.40.1 Phil Wang 2022-12-02 09:22:08 -08:00
  • 07c513e1f4 add vit with patch dropout, fully embrace structured dropout as multiple papers are now corroborating each other 0.40.0 Phil Wang 2022-12-02 09:10:42 -08:00
  • 2f87c0cf8f offer 1d versions, in light of https://arxiv.org/abs/2211.14730 0.39.1 Phil Wang 2022-12-01 10:31:05 -08:00
  • b6096b63a2 offer 1d versions, in light of https://arxiv.org/abs/2211.14730 0.39.0 Phil Wang 2022-12-01 10:28:11 -08:00
  • 59c8948c6a try to fix tests Phil Wang 2022-10-29 11:44:17 -07:00
  • cb6d749821 add a 3d version of cct, addressing https://github.com/lucidrains/vit-pytorch/issues/238 0.38.1 Phil Wang 2022-10-29 11:35:06 -07:00
  • ad1e6df5b3 add a 3d version of cct, addressing https://github.com/lucidrains/vit-pytorch/issues/238 0.38.1 Phil Wang 2022-10-29 11:34:28 -07:00
  • 61450ae1cf add a 3d version of cct, addressing https://github.com/lucidrains/vit-pytorch/issues/238 0.38.0 Phil Wang 2022-10-29 11:33:17 -07:00
  • 6ec8fdaa6d make sure global average pool can be used for vivit in place of cls token 0.37.1 Phil Wang 2022-10-24 19:59:48 -07:00
  • 13fabf901e add vivit Phil Wang 2022-10-24 09:34:04 -07:00
  • df46165f1f add vivit 0.37.0 Phil Wang 2022-10-24 09:26:40 -07:00
  • c0eb4c0150 Improving Readability (#220) Ryan Russell 2022-10-17 12:42:45 -05:00
  • 5f1a6a05e9 release updated mae where one can more easily visualize reconstructions, thanks to @Vishu26 0.36.2 Phil Wang 2022-10-17 10:41:46 -07:00
  • 9a95e7904e Update mae.py (#242) Srikumar Sastry 2022-10-17 12:41:10 -05:00
  • b4853d39c2 add the 3d simple vit 0.36.1 Phil Wang 2022-10-16 20:45:30 -07:00
  • 29fbf0aff4 begin extending some of the architectures over to 3d, starting with basic ViT 0.36.0 Phil Wang 2022-10-16 15:31:52 -07:00
  • 4b8f5bc900 add link to Flax translation by @conceptofmind Phil Wang 2022-07-27 08:58:18 -07:00
  • f86e052c05 offer way for extractor to return latents without detaching them v0.35.8 Phil Wang 2022-07-16 16:22:40 -07:00
  • 2fa2b62def slightly more clear of einops rearrange for cls token, for https://github.com/lucidrains/vit-pytorch/issues/224 v0.35.7 Phil Wang 2022-06-30 08:11:17 -07:00
  • 9f87d1c43b follow @arquolo feedback and advice for MaxViT v0.35.6 Phil Wang 2022-06-29 08:53:09 -07:00
  • 2c6dd7010a fix hidden dimension in MaxViT thanks to @arquolo v0.35.5 Phil Wang 2022-06-24 23:28:35 -07:00
  • 6460119f65 be able to accept a reference to a layer within the model for forward hooking and extracting the embedding output, for regionvit to work with extractor v0.35.4 Phil Wang 2022-06-19 08:22:18 -07:00
  • 4e62e5f05e make extractor flexible for layers that output multiple tensors, show CrossViT example v0.35.3 Phil Wang 2022-06-19 08:11:41 -07:00
  • b3e90a2652 add simple vit, from https://arxiv.org/abs/2205.01580 Phil Wang 2022-05-03 20:24:14 -07:00
  • ef8c0ac8bc add simple vit, from https://arxiv.org/abs/2205.01580 0.35.2 Phil Wang 2022-05-03 19:44:22 -07:00
  • c1b6cfde24 add simple vit, from https://arxiv.org/abs/2205.01580 0.35.1 Phil Wang 2022-05-03 19:36:08 -07:00
  • 4c37586510 add simple vit, from https://arxiv.org/abs/2205.01580 0.35.0 Phil Wang 2022-05-03 19:33:48 -07:00
  • 4ef72fc4dc add EsViT, by popular request, an alternative to Dino that is compatible with efficient ViTs with accounting for regional self-supervised loss Phil Wang 2022-05-03 10:29:29 -07:00
  • 70284c01b7 add EsViT, by popular request, an alternative to Dino that is compatible with efficient ViTs with accounting for regional self-supervised loss 0.34.1 Phil Wang 2022-05-03 10:21:41 -07:00
  • 0b347973fe add EsViT, by popular request, an alternative to Dino that is compatible with efficient ViTs with accounting for regional self-supervised loss 0.34.0 Phil Wang 2022-05-03 10:18:36 -07:00
  • c2aab05ebf fix bibtex typo (#212) Zhengzhong Tu 2022-04-07 05:15:05 +00:00
  • 81661e3966 fix mbconv residual block 0.33.2 Phil Wang 2022-04-06 16:43:06 -07:00
  • 13f8e123bb fix maxvit - need feedforwards after attention 0.33.1 Phil Wang 2022-04-06 16:34:40 -07:00
  • 2d4089c88e link to maxvit in readme Phil Wang 2022-04-06 16:24:12 -07:00
  • c7bb5fc43f maxvit intent to build (#211) 0.33.0 Phil Wang 2022-04-06 16:12:17 -07:00
  • 946b19be64 sponsor button Phil Wang 2022-04-06 14:12:11 -07:00
  • d93cd84ccd let windowed tokens exchange information across heads a la talking heads prior to pointwise attention in sep-vit 0.32.2 Phil Wang 2022-03-31 15:22:24 -07:00
  • 5d4c798949 cleanup sepvit 0.32.1 Phil Wang 2022-03-31 14:35:11 -07:00
  • d65a742efe intent to build (#210) 0.32.0 Phil Wang 2022-03-31 14:30:23 -07:00
  • 8c54e01492 do not layernorm on last transformer block for scalable vit, as there is already one in mlp head 0.31.1 Phil Wang 2022-03-31 13:25:21 -07:00
  • df656fe7c7 complete learnable memory ViT, for efficient fine-tuning and potentially plays into continual learning 0.30.1 Phil Wang 2022-03-31 09:51:12 -07:00
  • 4e6a42a0ca correct need for post-attention dropout Phil Wang 2022-03-30 10:50:57 -07:00
  • 64aae4680b correct need for post-attention dropout 0.30.0 Phil Wang 2022-03-30 10:05:19 -07:00
  • 6d7298d8ad link to tensorflow2 translation by @taki0112 Phil Wang 2022-03-28 09:05:34 -07:00
  • 9cd56ff29b CCT allow for rectangular images 0.29.1 Phil Wang 2022-03-26 14:02:49 -07:00
  • 2aae406ce8 add proposed parallel vit from facebook ai for exploration purposes Phil Wang 2022-03-23 10:42:35 -07:00
  • 9bae62f69d add proposed parallel vit from facebook ai for exploration purposes 0.29.0 Phil Wang 2022-03-23 10:31:38 -07:00
  • c2b2db2a54 fix window size of none for scalable vit for rectangular images 0.28.2 Phil Wang 2022-03-22 17:37:59 -07:00
  • 719048d1bd some better defaults for scalable vit 0.28.1 Phil Wang 2022-03-22 17:19:58 -07:00
  • d27721a85a add scalable vit, from bytedance AI 0.28.0 Phil Wang 2022-03-22 17:02:47 -07:00
  • cb22cbbd19 update to einops 0.4, which is torchscript jit friendly 0.27.1 Phil Wang 2022-03-22 13:58:00 -07:00
  • 6db20debb4 add patch merger 0.27.0 Phil Wang 2022-03-01 16:50:17 -08:00
  • 1bae5d3cc5 allow for rectangular images for efficient adapter 0.26.7 Phil Wang 2022-01-31 08:55:31 -08:00
  • 25b384297d return None from extractor if no attention layers 0.26.6 Phil Wang 2022-01-28 17:49:58 -08:00
  • 64a07f50e6 epsilon should be inside square root 0.26.5 Phil Wang 2022-01-24 17:24:41 -08:00
  • 126d204ff2 fix block repeats in readme example for Nest Phil Wang 2022-01-22 21:32:53 -08:00