Ever wanted to create something cool based on whatever LEGO bricks that you currently have, but stumped on ideas? Researchers at Carnegie Mellon University have you covered.
The team, which is made up of six individuals, has created an AI tool aptly called LegoGPT, which can generate complete LEGO designs based on text descriptions. The system is capable of creating original and valid structures such as chairs, ships, guitars, or even bookshelves, simply from written prompts.
According to the team’s research paper, LegoGPT was trained on a dataset of over 47,000 LEGO models covering more than 28,000 unique 3D objects. The model can produce fully buildable LEGO creations that meet physical constraints like stability and connectivity. The final output is designed to be realistic and buildable by humans and even robots.
The system ensures that no bricks overlap or float unrealistically by performing checks at every step of the building process. If a design becomes unstable, LegoGPT automatically rolls back to a previously stable state and continues generating from there. The result is a complete, coherent LEGO model that reflects the original text input.
The team claims LegoGPT can also be paired with image recognition tools. For instance, users could upload a photo of their existing LEGO pieces and let the AI suggest custom designs based on those bricks.
However, there are limitations. In its current form, LegoGPT can only generate designs within a 20 x 20 x 20 grid and is restricted to eight basic brick types. The good news is that the research team is actively working to expand the system’s capabilities, paving the way for more complex and advanced designs in the future.
LegoGPT and its resources are available publicly on Github. Additionally, users can interact directly with its live demo via HuggingFace, where it’s also freely available.
(Source: The Register)
