4 Commits

Author	SHA1	Message	Date
lucidrains	e300cdd7dc	fix multiheaded qk rmsnorm in nViT	2024-10-10 19:15:17 -07:00
Phil Wang	36ddc7a6ba	go all the way with the normalized vit, fix some scales	2024-10-10 10:42:37 -07:00
Phil Wang	74b62009f8	go for multi-headed rmsnorm for the qknorm on hypersphere vit	2024-10-10 08:09:58 -07:00
Phil Wang	f50d7d1436	add a hypersphere vit, adapted from https://arxiv.org/abs/2410.01131	2024-10-09 07:32:25 -07:00