Skip to content

The code associated with my cross-axis transformer research project.

Notifications You must be signed in to change notification settings

ElleLeonne/Cross-Axis-Transformer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Cross Axis Transformer with 2d Rotary Embeddings

See the paper

Capture

WandB tests & ablations

The code associated with my cross-axis transformer project.

The results of my last two years of private study, research, and code curation. All work described below was performed either solo, or with the help of chatbot assistants.

The model is novel and was theorized, designed, and coded by hand. The trainer was built by hand on top of the Lightning codebase. The attached paper was written in latex by hand.

AI was used for the initial learning of latex, however the contents of the paper were written solo and is not AI generated in any capacity.

About

The code associated with my cross-axis transformer research project.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages