99 Commits

Author	SHA1	Message	Date
Phil Wang	976f489230	add some tests	2021-12-22 09:13:31 -08:00
Phil Wang	2c368d1d4e	add extractor wrapper	2021-12-21 11:11:39 -08:00
Phil Wang	b983bbee39	release MobileViT, from @murufeng	2021-12-21 10:22:59 -08:00
Phil Wang	365b4d931e	add adaptive token sampling paper	2021-12-03 19:52:40 -08:00
Phil Wang	ff44d97cb0	make initial channels customizable for PiT	2021-11-22 18:08:49 -08:00
Phil Wang	b69b5af34f	dynamic positional bias for crossformer the more efficient way as described in appendix of paper	2021-11-22 17:39:36 -08:00
Phil Wang	36e32b70fb	complete and release crossformer	2021-11-22 17:10:53 -08:00
Phil Wang	5ae555750f	add SimMIM	2021-11-21 15:50:19 -08:00
Phil Wang	e8f6d72033	release masked autoencoder	2021-11-12 20:08:48 -08:00
Phil Wang	cb1729af28	more efficient feedforward for regionvit	2021-11-07 17:18:59 -08:00
Phil Wang	06d375351e	add RegionViT paper	2021-11-07 09:47:28 -08:00
Phil Wang	f196d1ec5b	move freqs in RvT to linspace	2021-10-05 09:23:44 -07:00
Phil Wang	d2d6de01d3	0.20.7	2021-08-30 08:14:43 -07:00
Phil Wang	6e1be11517	0.20.6	2021-08-21 09:03:54 -07:00
Phil Wang	aeed3381c1	use hardswish for levit	2021-08-19 08:22:55 -07:00
Phil Wang	3f754956fb	remove last transformer layer in t2t	2021-08-14 08:06:23 -07:00
Phil Wang	918869571c	fix hard distillation, thanks to @CiaoHe	2021-08-12 08:40:57 -07:00
Phil Wang	e5324242be	fix wrong norm in nest	2021-08-05 12:55:48 -07:00
Phil Wang	22da26fa4b	fix recorder in data parallel situation	2021-07-08 10:15:07 -07:00
Phil Wang	a6c085a2df	0.20.0 for cct	2021-07-02 15:48:48 -07:00
Phil Wang	64a2ef6462	fix mpp	2021-06-16 16:46:32 -07:00
Phil Wang	53884f583f	0.19.5	2021-06-16 14:24:46 -07:00
Phil Wang	60ad4e266e	layernorm on channel dimension == instancenorm2d with affine set to true	2021-06-03 16:41:45 -07:00
Phil Wang	a254a0258a	fix typo	2021-06-01 07:33:00 -07:00
Phil Wang	26df10c0b7	fix max pool in nest	2021-05-28 11:06:02 -07:00
Phil Wang	17cb8976df	make nest resilient to dimension that are not divisible by number of heads	2021-05-27 22:41:07 -07:00
Phil Wang	daf3abbeb5	add NesT	2021-05-27 22:02:17 -07:00
Phil Wang	b483b16833	0.18.4	2021-05-18 14:40:33 -07:00
Phil Wang	679e5be3e7	apply scale to 2d rel pos bias in levit	2021-05-10 11:37:23 -07:00
Phil Wang	04f86dee3c	implement SOTA new self-supervised learning technique from facebook for vision transformers, Dino	2021-05-02 14:00:36 -07:00
Phil Wang	6549522629	be able to accept non-square patches, thanks to @FilipAndersson245	2021-05-01 20:04:41 -07:00
Phil Wang	65bb350e85	0.17.2	2021-04-30 06:44:54 -07:00
Phil Wang	7807f24509	fix small bug	2021-04-29 15:39:41 -07:00
Phil Wang	30a1335d31	release twins svt	2021-04-29 14:55:25 -07:00
Phil Wang	4f3dbd003f	for PiT, project to increased dimensions on first grouped conv for depthwise-conv	2021-04-29 12:41:00 -07:00
Phil Wang	0df1505662	add zeroing of weight parameters of batchnorm in levit just before residual connection, noticed by @EelcoHoogendoorn	2021-04-27 08:41:16 -07:00
Phil Wang	3df6c31c61	fix norm issues in cvt	2021-04-27 08:36:17 -07:00
Phil Wang	54af220930	fix cvt	2021-04-26 20:37:51 -07:00
Phil Wang	bad4b94e7b	fix all issues with rotary vision transformer	2021-04-25 12:09:32 -07:00
Phil Wang	e42e9876bc	offer a way to turn off ds conv in rotary vision transformer for ablation	2021-04-20 10:12:03 -07:00
Phil Wang	566365978d	add ability to turn off rotary, for ablation	2021-04-20 09:00:27 -07:00
Phil Wang	34f78294d3	fix pooling bugs across a few new archs	2021-04-19 22:36:23 -07:00
Phil Wang	4c29328363	fix frequency in rotary vision transformer	2021-04-15 16:06:32 -07:00
Phil Wang	27ac10c1f1	0.16.3	2021-04-14 16:53:05 -07:00
Phil Wang	1d8b7826bf	update personal pet vit	2021-04-14 15:56:39 -07:00
Phil Wang	6289619e3f	0.16.1	2021-04-14 08:05:08 -07:00
Phil Wang	30b37c4028	add LocalViT	2021-04-12 19:17:32 -07:00
Phil Wang	4497f1e90f	add rotary vision transformer	2021-04-10 22:59:15 -07:00
Phil Wang	b50d3e1334	cleanup levit	2021-04-06 13:46:19 -07:00
Phil Wang	e075460937	stray print	2021-04-06 13:38:52 -07:00