Whether you have extensive 3D modeling skills, just dabble in it, or have never even attempted 3D modeling…artificial intelligence (AI) has made generating 3D models available for anyone. These 3D generation AI technologies will allow for users to make 3D models without the need for special training.

While this technology makes creating 3D modeling available to anyone, it will be a great tool for designers to use as a tool in their workflow. Generative AI for 3D models aid in the conceptualization of complex components and shapes, and can do so quickly, saving time. These models are high-quality, relightable, exportable, and can be further processed in common 3D tools.

Let’s explore three AI 3D model generating tools: text-to-3D, image-to-3D, and sketch-to-3D.

Text to 3D Model

A text-to-3D model is a machine learning model in which a user inputs a text into a prompt and a 3D model is produced matching that description.

Text-to-3D Model Apps:

Magic3D

Nvidia’s Magic 3D will offer users the ability to create high-quality 3D models from inputted text prompts. This technology is still in development.*

DreamFusion

Similar to Magic3D, Google’s DreamFusion also uses AI to generate 3D models from text prompts. It can not only generate any simple object, but also combine that object with another (such as: “dinosaur playing frisbee”). Google has not yet provided public access to the model.*

PointE

OpenAI, the creator of the well known AI visual art tool Dall-E, open sourced Point-E, a machine learning system that allows 3D modeling in minutes. Using a text prompt, Point-E produces point clouds, or 3D models that are made up of discrete groupings of data in space.

Point-E has not yet been officially launched through OpenAI*, but it is available via Github for those more technically minded. Check out the Point-E demo here: https://huggingface.co/spaces/openai/point-e where they use “a small, worse quality text-to-3D model to produce 3D point clouds directly from text descriptions.” Disclaimer: Keep your expectations low (ie. see text-to-3D “duck” below–duck or a couple of lemons?).

3DFY

3DFY Prompt is a new text to 3D model for 3DFY.ai Using generative AI, users will be able to create high-quality 3D models from text. 3DFY Prompt will be launching soon.* Visit http://www.3dfy.ai to join the waitlist. Side note: their website is pretty cool and worth checking out.

Image-to-3D

This technology turns 2D image inputs into 3D models.

Image-to-3D model apps:

Kaedim

Kaedim enables users to create 3D art with nothing more than an image in a quick, cost effective way, as well as add texturing and color. Zero 3D modeling experience is required. Kaedim’s 3D asset generation can be seamlessly integrated with your favorite 3D tools. Visit http://www.kaedim3d.com and start a trial or explore monthly plans.

GET3D

Another 3D modeling app from Nvidia, GET3D, generates high-quality 3D shapes that are learned from images. It can generate up to 20 3D shapes per second that combine to form textured 3D models that can be refined and further processed in common 3D workflows. GET3D is available as an open-source model on Github.

Sketch to 3D

Sketch to 3D turns your 2D drawings into 3D models.

Sketch to 3D model apps:

Plasmo

A 3D design app that automatically turns your sketches into 3D models, and helps you build better products faster. You can even sketch right within the app. Plasmo integrates seamlessly with your existing workflow. Try Plasmo for free at http://www.Plasmo.ai.

3D File Formats that can be exported using the AI App(s)

.OBJ (OBJect)

.FBX (Filmbox)

.gITF (GL Transmission Format)

.STL (Stereolithography)

.PLY (Polygon File Format)

.DAE (Digital Asset Exchange)

Conclusion

AI generated 3D modeling is still in the beginning stages, with most models mentioned above still in development and beta phases. Once this technology is refined, it will help speed up processes and development for 3D designers, video games, virtual reality, and other areas in which 3D is often used.

*All release date references are based on a publishing date of February, 2023.