huggingface / transformers

State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

125.0k

Python

Architecture: library

View Original Source

main

transformers

docs

src

transformers

models

bert

gpt2

llama

pipelines

trainer.py

tests

setup.py

Structural Architecture

Transformers manages unprecedented scope by unifying PyTorch, TF, and JAX frameworks under incredibly strict, deeply nested model class structures. Code duplication is explicitly favored over abstraction to guarantee that reading one model's file teaches everything about it.

Anti-abstraction: Models deliberately copy/paste layers to avoid deep tangled inheritance.
Multi-framework execution dynamically mapped at runtime through model config abstractions.

Notable Directories

/src/transformers/models

The absolute nexus housing hundreds of discrete LLM/CNN model architectures.

/tests

Vast test suites validating exact precision across tensor gradients in multiple backend frameworks.

/docs

Rigorous documentation generation tools.

Build Your Own Ecosystem

Learn from transformers's structure and craft your own software scaffolding.