Animator Lyndon Barrois creates new worlds with Sora
Filmmaker Lyndon Barrois shares how he uses Sora as a storytelling tool.
Filmmaker Lyndon Barrois shares how he uses Sora as a storytelling tool.
OpenAI publishes the system card for its video generation model Sora. Sora can accept text, image, and video inputs to generate new videos. Its design draws on lessons from DALL-E and GPT models, aiming to provide richer tools for storytelling and creative expression.
OpenAI has simplified, stabilized, and scaled continuous-time consistency models, achieving sample quality comparable to mainstream diffusion models with only two sampling steps.
OpenAI proposes Consistency Models, which can generate images, audio, and video without relying on iterative sampling. While diffusion models have advanced these fields, their iterative sampling process results in slow generation speeds.
OpenAI publishes a detailed explanation of the technical principles and safety research behind its text-to-speech model Voice Engine. Voice Engine can generate speech from text, and the official team also introduces safety research conducted around this model.
Artificial Analysis launches a text-to-image leaderboard and Image Arena, calculating ELO scores based on over 45,000 human image preference votes, covering mainstream open-source and closed-source models including Midjourney, DALL·E, Stable Diffusion, and Playground.
Canva is a visual communication platform with over 175 million monthly active users, used for creating presentations, videos, documents, websites, and social media graphics. Facing the challenge that most knowledge workers globally lack design training, Canva enables anyone to create visually appealing content through its user-friendly interface, extensive asset library, and time-saving tools.
OpenAI releases its new flagship model GPT-4o, capable of real-time reasoning across audio, vision, and text.
OpenAI launches new technology to help researchers identify content generated by its tools, while joining the Coalition for Content Provenance and Authenticity (C2PA) Steering Committee to advance industry standards.
OpenAI shares lessons learned from the small-scale preview of Voice Engine, a custom voice model that can create tailored voices from text. The model is currently in limited testing.
After OpenAI released Sora last month, it partnered with artists to explore its applications in creative workflows.
OpenAI explores large-scale training of generative models on video data, jointly training text-conditioned diffusion models on videos and images of varying durations, resolutions, and aspect ratios, using a transformer architecture that operates on spatiotemporal patches of video and image latent codes. Its largest model, Sora, can generate one-minute high-fidelity videos. Results indicate that scaling video generation models is a promising path toward building general-purpose physical world simulators.
Why it matters: OpenAI publicly reveals Sora's technical roadmap for the first time, explaining how video generation models can be scaled into physical world simulators.
OpenAI releases GPT-4 Turbo at DevDay, supporting 128K context with lower pricing and new vision capabilities. It also launches developer products including the Assistants API and DALL·E 3 API.
OpenAI announced that DALL·E 3 is now available in ChatGPT Plus and Enterprise. To prepare for a broader release, OpenAI developed a set of safety mitigations and also shared its progress on content provenance research.
Why it matters: DALL·E 3 is open to ChatGPT Plus and Enterprise, and the official announcement also explains the safety mitigations made for the expanded release.
OpenAI has acquired Global Illumination, and all members of the team have joined OpenAI.
Hugging Face's Diffusers library has reached its one-year anniversary, having expanded from text-to-image to video, 3D, and image editing. The library has added support for models such as DeepFloyd IF, SDXL, VideoFusion, Text2Video-Zero, and OpenAI Shap-E, and integrates LoRA, Torch 2.0 optimizations, and safety checking components.
OpenAI fine-tuned GPT-3 to drive and scale "done-for-you" video creation. The original text does not disclose the specific version number, parameter scale, or availability information.
OpenAI is using GPT-3 to create a new generation of AI-driven characters.
OpenAI announces that the DALL·E API has now entered public beta, and developers can start building applications with it as of today.
OpenAI announces that DALL·E has removed its waitlist, and new users can start creating right away. The company says lessons learned from deployment and improvements to its safety systems have made broader access possible.
OpenAI announces it has trained and open-sourced Whisper, a speech recognition neural network, stating that its robustness and accuracy in English speech recognition approach human level.
Why it matters: OpenAI open-sources the speech recognition model Whisper, whose English recognition robustness and accuracy approach human level.
OpenAI introduces outpainting for DALL·E, which can extend images to any size to tell bigger stories. The feature lets users continue generating content beyond the original frame, expanding creative space.
OpenAI announces that DALL·E has entered beta and will invite 1 million people from the waitlist over the coming weeks. Users can generate images with free credits that reset monthly, or purchase additional credits at $15 for 115 generations.
Why it matters: DALL·E moves from waitlist to million-scale access, with free credits and paid add-ons, offering a way to gauge the accessibility threshold of early image generation tools.
OpenAI deployed a new technique for DALL·E 2 so that the people images it generates more accurately reflect the diversity of the world's population, reducing bias and improving safety.
OpenAI's DALL·E 2 research preview has attracted more than 3,000 artists from over 118 countries to incorporate it into their creative workflows. Artists in the early access group helped discover new uses for DALL·E and provided key input on feature decisions.
To open DALL·E 2 to the public, OpenAI adopted mitigations during pretraining to reduce the risks posed by powerful image generation models. To this end, the team set up multiple safeguards to prevent generated images from violating its content policy.
This Hugging Face blog post takes an in-depth look at denoising diffusion probabilistic models (DDPM), based on the 2020 paper by Ho et al., with a step-by-step PyTorch implementation whose code references Phil Wang's implementation. The article covers both the forward noising and reverse denoising processes; the DDPM authors use T=1000 time steps, encode time steps with sinusoidal positional embeddings, and use Wide ResNet blocks as the core module of the U-Net.
OpenAI updates progress on the DALL·E 2 research preview: early users have generated over 3 million images to date and have helped improve safety processes. OpenAI says it will begin adding up to 1,000 new users from the waitlist each week.
Why it matters: Progress on user scale and safety processes during the DALL·E 2 research preview phase, offering insight into its early rollout pace.
OpenAI trained a neural network called DALL·E that can generate images from natural language text descriptions, covering a wide range of concepts.
OpenAI released an organizational structure update, stating that the company has undergone dramatic change and growth over the past year. The original text did not disclose the specific adjustments or the personnel involved.
OpenAI found that a large Transformer model trained on language can also generate coherent image completions and samples when trained instead on sequences of pixels. The research also established a correlation between sample quality and image classification accuracy, showing that its best generative model contains features under an unsupervised setting that are comparable to those of top convolutional networks.
OpenAI introduced the neural network Jukebox, which can generate songs in the style of various genres and artists as raw audio, including rudimentary singing. The team also open-sourced the model weights and code, and provided a tool for listening to generated samples.
OpenAI introduced the deep neural network MuseNet, which can generate 4-minute musical compositions featuring 10 instruments and can blend a variety of styles ranging from country to Mozart to the Beatles. MuseNet was not explicitly programmed with musical rules; instead, by learning to predict the next token across hundreds of thousands of MIDI files, it discovered harmony, rhythm, and style patterns on its own. It uses the same general-purpose unsupervised technique as GPT-2, namely a large-scale transformer model.
OpenAI developed the Sparse Transformer, a deep neural network that set records on the task of predicting the next item in sequences of text, images, or sound. By improving the algorithm of the attention mechanism, it can extract patterns from sequences 30 times longer than before.
OpenAI proposed the invertible generative model Glow, which uses invertible 1x1 convolutions and simplifies the architecture of previous invertible generative models. The model can generate realistic high-resolution images, supports efficient sampling, and can discover features used to manipulate data attributes. OpenAI also open-sourced the model code and provided an online visualization tool.
OpenAI announced that new donors are joining its ranks of supporters. The original text did not disclose details such as the donors' identities, amounts, or specific uses.
OpenAI published an article explaining the orientation of its "special projects": impactful scientific work requires choosing the right problems—not merely interesting ones, but ones whose solutions truly matter.
OpenAI announced that two new members, Pieter and Shivon, are joining the team, the two latest team personnel updates it has announced.