31 Commits

Author	SHA1	Message	Date
Phil Wang	5db8d9deed	update readme about non-square images	2021-01-12 06:55:45 -08:00
Phil Wang	e8ca6038c9	allow for DistillableVit to still run predictions	2021-01-11 10:49:14 -08:00
Phil Wang	1106a2ba88	link to official repo	2021-01-08 08:23:50 -08:00
Phil Wang	f95fa59422	link to resources for vision people	2021-01-04 10:10:54 -08:00
Phil Wang	be1712ebe2	add quote	2020-12-28 10:22:59 -08:00
Phil Wang	1a76944124	update readme	2020-12-27 19:10:38 -08:00
Phil Wang	74074e2b6c	offer easy way to turn DistillableViT to ViT at the end of training	2020-12-25 11:16:52 -08:00
Phil Wang	e0007bd801	add distill diagram	2020-12-24 11:34:15 -08:00
Phil Wang	dc4b3327ce	no grad for teacher in distillation	2020-12-24 11:11:58 -08:00
Phil Wang	aa8f0a7bf3	Update README.md	2020-12-24 10:59:03 -08:00
Phil Wang	34e6284f95	Update README.md	2020-12-24 10:58:41 -08:00
Phil Wang	aa9ed249a3	add knowledge distillation with distillation tokens, in light of new finding from facebook ai	2020-12-24 10:39:15 -08:00
Phil Wang	ea0924ec96	update readme	2020-12-23 19:06:48 -08:00
Phil Wang	24339644ca	offer a way to use mean pooling of last layer	2020-12-23 17:23:58 -08:00
Phil Wang	b786029e18	fix the dimension per head to be independent of dim and heads, to make sure users do not have it be too small to learn anything	2020-12-17 07:43:52 -08:00
Phil Wang	a656a213e6	update diagram	2020-12-04 12:26:28 -08:00
Long M. Lưu	3f50dd72cf	Update README.md	2020-11-21 18:37:03 +07:00
Phil Wang	4f84ad7a64	authors are now known	2020-11-03 14:28:20 -08:00
Phil Wang	c74bc781f0	cite	2020-11-03 11:59:05 -08:00
Phil Wang	c1043ab00c	update readme	2020-10-26 19:01:03 -07:00
Phil Wang	5b5d98a3a7	dropouts are more specific and aggressive in the paper, thanks for letting me know @hila-chefer	2020-10-14 09:22:16 -07:00
Phil Wang	0b2b3fc20c	add dropouts	2020-10-13 13:11:59 -07:00
Phil Wang	b298031c17	write up example for using efficient transformers	2020-10-07 19:15:21 -07:00
Phil Wang	f7123720c3	add masking	2020-10-07 11:21:03 -07:00
Phil Wang	8fb261ca66	fix a bug and add suggestion for BYOL pre-training	2020-10-04 14:55:29 -07:00
Phil Wang	112ba5c476	update with link to Yannics video	2020-10-04 13:53:47 -07:00
Phil Wang	f899226d4f	add diagram	2020-10-04 12:47:08 -07:00
Phil Wang	ee8088b3ea	first commit	2020-10-04 12:35:01 -07:00
Phil Wang	ea03db32f0	Update README.md	2020-10-03 15:49:27 -07:00
Phil Wang	30362d50dc	Update README.md	2020-10-03 15:49:02 -07:00
Phil Wang	efb40e0b01	Initial commit	2020-10-03 15:47:26 -07:00