Apr 10, 2024
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Posted by Cecile G. Tamura in category: futurism
Eagle and Finch.
RWKV with Matrix-Valued States and Dynamic Recurrence.
We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture.
Continue reading “Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence” »