134 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Phil Wang	04f86dee3c	implement SOTA new self-supervised learning technique from facebook for vision transformers, Dino 0.18.1	2021-05-02 14:00:36 -07:00
Phil Wang	6549522629	be able to accept non-square patches, thanks to @FilipAndersson245 0.17.3	2021-05-01 20:04:41 -07:00
Phil Wang	6a80a4ef89	update readme	2021-05-01 11:51:35 -07:00
Phil Wang	9f05587a7d	0.17.2 0.17.2	2021-04-30 06:44:59 -07:00
Phil Wang	65bb350e85	0.17.2	2021-04-30 06:44:54 -07:00
Phil Wang	fd4a7dfcf8	Merge pull request #102 from jon-tow/rvt-add-use-glu-flag ... Add `use_glu` flag to `RvT`	2021-04-30 06:44:41 -07:00
Jonathan Tow	6f3a5fcf0b	Add use_glu flag to RvT	2021-04-30 02:07:41 -04:00
Phil Wang	7807f24509	fix small bug	2021-04-29 15:39:41 -07:00
Phil Wang	a612327126	readme	2021-04-29 15:22:12 -07:00
Phil Wang	30a1335d31	release twins svt	2021-04-29 14:55:25 -07:00
Phil Wang	ab781f7ddb	add Twins SVT (small)	2021-04-29 14:54:06 -07:00
Phil Wang	4f3dbd003f	for PiT, project to increased dimensions on first grouped conv for depthwise-conv 0.16.13	2021-04-29 12:41:00 -07:00
Phil Wang	60b5687a79	cleanup rvt	2021-04-27 11:45:46 -07:00
Phil Wang	0df1505662	add zeroing of weight parameters of batchnorm in levit just before residual connection, noticed by @EelcoHoogendoorn 0.16.12	2021-04-27 08:41:16 -07:00
Phil Wang	3df6c31c61	fix norm issues in cvt 0.16.11	2021-04-27 08:36:17 -07:00
Phil Wang	54af220930	fix cvt 0.16.10	2021-04-26 20:37:51 -07:00
Phil Wang	bad4b94e7b	fix all issues with rotary vision transformer 0.16.9	2021-04-25 12:09:32 -07:00
Phil Wang	fbced01fe7	cite	2021-04-20 18:36:54 -07:00
Phil Wang	e42e9876bc	offer a way to turn off ds conv in rotary vision transformer for ablation 0.16.8	2021-04-20 10:12:03 -07:00
Phil Wang	566365978d	add ability to turn off rotary, for ablation 0.16.7	2021-04-20 09:00:27 -07:00
Phil Wang	34f78294d3	fix pooling bugs across a few new archs 0.16.6	2021-04-19 22:36:23 -07:00
Phil Wang	4c29328363	fix frequency in rotary vision transformer 0.16.4	2021-04-15 16:06:32 -07:00
Phil Wang	27ac10c1f1	0.16.3 0.16.3	2021-04-14 16:53:05 -07:00
Phil Wang	fa216c45ea	tweak	2021-04-14 16:52:53 -07:00
Phil Wang	1d8b7826bf	update personal pet vit 0.16.2	2021-04-14 15:56:39 -07:00
Phil Wang	53b3af05f6	use convolution on query with padding to give the network absolute spatial awareness in addition to relative encoding from rotary embeddings	2021-04-14 15:56:02 -07:00
Phil Wang	6289619e3f	0.16.1 0.16.1	2021-04-14 08:05:08 -07:00
Phil Wang	b42fa7862e	Merge pull request #91 from shabie/patch-1 ... Fix alpha coefficient multiplication in the loss	2021-04-14 08:04:50 -07:00
shabie	dc6622c05c	Fix alpha coefficient multiplication in the loss	2021-04-14 11:36:43 +02:00
Phil Wang	30b37c4028	add LocalViT 0.16.0	2021-04-12 19:17:32 -07:00
Phil Wang	4497f1e90f	add rotary vision transformer	2021-04-10 22:59:15 -07:00
Phil Wang	b50d3e1334	cleanup levit 0.15.2	2021-04-06 13:46:19 -07:00
Phil Wang	e075460937	stray print 0.15.1	2021-04-06 13:38:52 -07:00
Phil Wang	5e23e48e4d	Merge pull request #88 from lucidrains/levit ... fix images	2021-04-06 13:37:46 -07:00
Phil Wang	db04c0f319	fix images	2021-04-06 13:37:23 -07:00
Phil Wang	0f31ca79e3	Merge pull request #87 from lucidrains/levit ... levit without pos emb 0.15.0	2021-04-06 13:36:26 -07:00
Phil Wang	2cb6b35030	complete levit	2021-04-06 13:36:11 -07:00
Phil Wang	2ec9161a98	levit without pos emb	2021-04-06 12:58:05 -07:00
Phil Wang	3a3038c702	add layer dropout for CaiT 0.14.5	2021-04-01 20:30:37 -07:00
Phil Wang	b1f1044c8e	offer hard distillation as well 0.14.4	2021-04-01 16:56:14 -07:00
Phil Wang	deb96201d5	readme	2021-03-31 23:02:47 -07:00
Phil Wang	05b47cc070	make sure layerscale epsilon is a function of depth 0.14.2	2021-03-31 22:53:04 -07:00
Phil Wang	9ef8da4759	add CaiT, new vision transformer out of facebook AI, complete with layerscale, talking heads, and cls -> patch cross attention 0.14.1	2021-03-31 22:42:16 -07:00
Phil Wang	506fcf83a6	add documentation for three recent vision transformer follow-up papers	2021-03-31 09:22:15 -07:00
Phil Wang	6fb360a1ff	add arxiv links for now, document in readme later	2021-03-30 22:26:44 -07:00
Phil Wang	9332b9e8c9	cite	2021-03-30 22:16:14 -07:00
Phil Wang	da950e6d2c	add working PiT 0.12.0	2021-03-30 22:15:19 -07:00
Phil Wang	4b9a02d89c	use depthwise conv for CvT projections 0.11.1	2021-03-30 18:18:35 -07:00
Phil Wang	518924eac5	add CvT 0.11.0	2021-03-30 14:42:39 -07:00
Phil Wang	e712003dfb	add CrossViT 0.10.3	2021-03-30 00:53:27 -07:00