Introduction to Shap-E: Text-to-3D

Ng Wai Foong
5 min readMay 5

Generate 3D objects conditioned on text or images

Image by the author

By reading this article, you will learn to use Shap-E for 3D objects generation. As of 5 May 2023, OpenAI officially released Shap-E, which is

a system to generate 3D objects conditioned on text or images

Similar to its predecessor Point-E, Shap-E is capable of generating coherent 3D objects when conditioned on a rendering from a single viewpoint (image) or text prompt directly.

Shap-E contains the following models:

  • encoder — converts 3D assets into the parameters of small neural networks which represent the 3D shape and texture as an implicit function. The resulting implicit function can be rendered from arbitrary viewpoints or imported into downstream applications as a mesh.
  • latent diffusion generates novel implicit functions conditioned on either images or text descriptions. It produce latents which must be linearly projected to get the final implicit function parameters.

Both models are trained on the same datasets as Point-E with the following improvements:

  • rendering is based on 60 views of each model when computing point clouds. Previously, Point-E used 20 views and the final output is prone to small cracks
  • final output produces 16K points in each point cloud instead of 4k points used in Point-E
  • the lighting and material setup only include diffuse materials
  • datasets enlargement with a million more 3D assets and 120K captions from human annotators

Note that the generated 3D objects are typically lower fidelity than professional 3D assets and geared towards cartoonish assets.

Let’s proceed to the next section for setup and installation.


It is recommend to create a new virtual environment before you continue with the installation.

Activate it and clone the whole repository as follows:

Then, run the following command to install Shap-E:

cd shap-e
pip install -e .
Ng Wai Foong

Senior AI Engineer@Yoozoo | Content Writer #NLP #datascience #programming #machinelearning | Linkedin: