CSG-based ML-supported 3D translation of sketches into game assets for game designers

This research project aims to develop a new technique for translating 2D sketches into 3D assets like building models using machine learning and Constructive Solid Geometry (CSG) algorithms. The ability to quickly create 3D building models from simple freehand sketches has widespread applications in fields like architecture and construction. The key innovation is the use of CSG principles to understand geometric elements from sketches, extrude them into 3D, and combine simple shapes to create complex structures.
This allows the creation of building models with walls, windows, doors, and other features that closely match the original sketch, without requiring extensive 3D modeling expertise. Specialized sketch recognition methods, including machine learning techniques for edge detection and point decimation, are used to identify lines, arcs, and intersections, and transform them into the necessary CSG primitives. We plan to develop a user-friendly web interface where users can sketch freehand and see a live 3D model preview generated in real-time by reconstructing surfaces from the identified geometric features. This integrated sketching and 3D modeling environment provides an intuitive workflow, harnessing CSG techniques to generate complex forms from sparse 2D input. Overall, this research aims to improve sketch-based 3D modeling methods and enable more users to create 3D content with ease. The researchers believe this work has the potential to significantly enhance productivity in construction planning, enable faster design iterations, and increase architectural creativity.