56 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Phil Wang	e8ca6038c9	allow for DistillableVit to still run predictions 0.6.5	2021-01-11 10:49:14 -08:00
Phil Wang	1106a2ba88	link to official repo	2021-01-08 08:23:50 -08:00
Phil Wang	f95fa59422	link to resources for vision people	2021-01-04 10:10:54 -08:00
Phil Wang	be1712ebe2	add quote	2020-12-28 10:22:59 -08:00
Phil Wang	1a76944124	update readme	2020-12-27 19:10:38 -08:00
Phil Wang	2263b7396f	allow distillable efficient vit to restore efficient vit as well 0.6.4	2020-12-25 19:31:25 -08:00
Phil Wang	74074e2b6c	offer easy way to turn DistillableViT to ViT at the end of training 0.6.3	2020-12-25 11:16:52 -08:00
Phil Wang	0c68688d61	bump for release	2020-12-25 09:30:48 -08:00
Phil Wang	5918f301a2	cleanup	2020-12-25 09:30:38 -08:00
Phil Wang	4a6469eecc	Merge pull request #51 from umbertov/main ... Add class for distillation with efficient attention	2020-12-25 09:21:17 -08:00
Umberto Valleriani	5a225c8e3f	Add class for distillation with efficient attention ... DistillableEfficientViT does the same as DistillableViT, except it may accept a custom transformer encoder, possibly implementing an efficient attention mechanism	2020-12-25 13:46:29 +01:00
Phil Wang	e0007bd801	add distill diagram	2020-12-24 11:34:15 -08:00
Phil Wang	db98ed7a8e	allow for overriding alpha as well on forward in distillation wrapper 0.6.2	2020-12-24 11:18:36 -08:00
Phil Wang	dc4b3327ce	no grad for teacher in distillation	2020-12-24 11:11:58 -08:00
Phil Wang	aa8f0a7bf3	Update README.md	2020-12-24 10:59:03 -08:00
Phil Wang	34e6284f95	Update README.md	2020-12-24 10:58:41 -08:00
Phil Wang	aa9ed249a3	add knowledge distillation with distillation tokens, in light of new finding from facebook ai 0.6.0	2020-12-24 10:39:15 -08:00
Phil Wang	ea0924ec96	update readme	2020-12-23 19:06:48 -08:00
Phil Wang	59787a6b7e	allow for mean pool with efficient version too 0.5.1	2020-12-23 18:15:40 -08:00
Phil Wang	24339644ca	offer a way to use mean pooling of last layer 0.5.0	2020-12-23 17:23:58 -08:00
Phil Wang	b786029e18	fix the dimension per head to be independent of dim and heads, to make sure users do not have it be too small to learn anything 0.4.0	2020-12-17 07:43:52 -08:00
Phil Wang	9624181940	simplify mlp head 0.3.0	2020-12-07 14:31:50 -08:00
Phil Wang	a656a213e6	update diagram	2020-12-04 12:26:28 -08:00
Phil Wang	f1deb5fb7e	Merge pull request #31 from minhlong94/main ... Update README and documentation	2020-11-21 08:05:38 -08:00
Long M. Lưu	3f50dd72cf	Update README.md	2020-11-21 18:37:03 +07:00
Long M. Lưu	ee5e4e9929	Update vit_pytorch.py	2020-11-21 18:23:04 +07:00
Phil Wang	6c8dfc185e	remove float(-inf) as masking value 0.2.7	2020-11-13 12:25:21 -08:00
Phil Wang	4f84ad7a64	authors are now known	2020-11-03 14:28:20 -08:00
Phil Wang	c74bc781f0	cite	2020-11-03 11:59:05 -08:00
Phil Wang	dc5b89c942	use einops repeat	2020-10-28 18:13:57 -07:00
Phil Wang	c1043ab00c	update readme	2020-10-26 19:01:03 -07:00
Phil Wang	7a214d7109	allow for training on different image sizes, provided images are smaller than what was passed as image_size keyword on init 0.2.6	2020-10-25 13:17:42 -07:00
Phil Wang	6d1df1a970	more efficient 0.2.5	2020-10-22 22:37:06 -07:00
Phil Wang	d65a8c17a5	remove dropout from last linear to logits 0.2.4	2020-10-16 13:58:23 -07:00
Phil Wang	f7c164d910	assert minimum number of patches 0.2.3	2020-10-16 12:19:50 -07:00
Phil Wang	c7b74e0bc3	rename ipy notebook	2020-10-14 10:35:46 -07:00
Phil Wang	5b5d98a3a7	dropouts are more specific and aggressive in the paper, thanks for letting me know @hila-chefer	2020-10-14 09:22:16 -07:00
Phil Wang	b0e4790c24	bump package 0.2.1	2020-10-13 13:12:19 -07:00
Phil Wang	0b2b3fc20c	add dropouts	2020-10-13 13:11:59 -07:00
Phil Wang	ced464dcb4	Update setup.py	2020-10-11 00:06:26 -07:00
Phil Wang	5bf45a2d4d	Merge pull request #4 from adimyth/main ... Image Classification Example	2020-10-10 19:12:31 -07:00
adimyth	fa32e22855	adds a classification example using 'cats & dogs' data	2020-10-11 03:15:19 +05:30
Phil Wang	a0fa41070f	norm cls token before sending to mlp head 0.2.0	2020-10-10 12:08:42 -07:00
Phil Wang	b298031c17	write up example for using efficient transformers	2020-10-07 19:15:21 -07:00
Phil Wang	d66b29e4cf	cleanup stray print 0.0.5	2020-10-07 11:22:45 -07:00
Phil Wang	f7123720c3	add masking 0.0.4	2020-10-07 11:21:03 -07:00
Phil Wang	f5fffd9e2e	remove extraneous line	2020-10-04 15:22:26 -07:00
Phil Wang	8fb261ca66	fix a bug and add suggestion for BYOL pre-training 0.0.3	2020-10-04 14:55:29 -07:00
Phil Wang	112ba5c476	update with link to Yannics video	2020-10-04 13:53:47 -07:00
Phil Wang	f899226d4f	add diagram	2020-10-04 12:47:08 -07:00