10 Commits

Author	SHA1	Message	Date
Jonathan Tow	6f3a5fcf0b	Add use_glu flag to RvT	2021-04-30 02:07:41 -04:00
Phil Wang	60b5687a79	cleanup rvt	2021-04-27 11:45:46 -07:00
Phil Wang	bad4b94e7b	fix all issues with rotary vision transformer	2021-04-25 12:09:32 -07:00
Phil Wang	e42e9876bc	offer a way to turn off ds conv in rotary vision transformer for ablation	2021-04-20 10:12:03 -07:00
Phil Wang	566365978d	add ability to turn off rotary, for ablation	2021-04-20 09:00:27 -07:00
Phil Wang	34f78294d3	fix pooling bugs across a few new archs	2021-04-19 22:36:23 -07:00
Phil Wang	4c29328363	fix frequency in rotary vision transformer	2021-04-15 16:06:32 -07:00
Phil Wang	fa216c45ea	tweak	2021-04-14 16:52:53 -07:00
Phil Wang	53b3af05f6	use convolution on query with padding to give the network absolute spatial awareness in addition to relative encoding from rotary embeddings	2021-04-14 15:56:02 -07:00
Phil Wang	4497f1e90f	add rotary vision transformer	2021-04-10 22:59:15 -07:00