OpenAI News·· 2019-04-23AI score6
OpenAI Proposes Sparse Transformer: Modeling Ultra-Long Sequences with Sparse Attention
Generative modeling with sparse transformers
AI briefing
OpenAI developed the Sparse Transformer, a deep neural network that set records on the task of predicting the next item in sequences of text, images, or sound. By improving the algorithm of the attention mechanism, it can extract patterns from sequences 30 times longer than before.
Open original openai.com