Transformer (machine learning model)

wikipedia:Transformer (machine learning model)

• GPT-3: the architecture is a decoder-only transformer network with a 2048-token-long context and 175 billion parameters, requiring 800GB to store.

Related

• Attention is all you need (2017)
• GPT: GPT-4, GPT-3
• Diffusion

See also

• Transformer, GPT
• GPT, GPT-2, GPT-3, GPT-4, Tiktoken, Bigram, Transformer, PaLM, ChatGPT
• Machine learning, Deep learning, AWS Sagemaker, PyTorch, Kubeflow, TensorFlow, Keras, Torch, Spark ML, Tinygrad, Apple Neural Engine, Scikit-learn, MNIST, MLOps, AutoML, ClearML, PostgresML, AWS Batch, Transformer, Diffusion, Backpropagation, JAX, Vector database
• OpenAI, GitHub Copilot, ChatGPT, OpenAI Codex, GPT-3, GPT-4, Whisper, Sam Altman, Mira Murati, Greg Brockman, Ilya Sutskever, OpenAI board