Infinite Canvas AI
Sketch to Image Generation
Draw rough sketches on an infinite canvas, and AI transforms them into polished images. TLDraw meets DALL-E 3.
Overview
A creative tool that combines freeform sketching with AI image generation. Draw a rough concept, describe what you want, and DALL-E 3 generates a polished image that matches your sketch's composition.
Challenge
Text-to-image AI is powerful but hard to control. Describing spatial relationships in words is awkward: "a house on the left with a tree on the right" often produces unexpected layouts. Sketching is the natural way to communicate composition, but most AI tools ignore it.
Approach
Integrated TLDraw as the canvas layer. Users can draw, add shapes, and annotate freely on an infinite zoomable surface.
Built a generation flow that captures the canvas region, converts it to an image, and sends it alongside the text prompt to DALL-E 3 as a composition reference.
Optimized for edge runtime with streaming responses. Generation feedback appears immediately, not after a loading screen.
Added Zustand for state management: canvas state, generation history, and user preferences persist across sessions.
Outcome
The sketch-to-image workflow dramatically improved composition control. Users could iterate quickly: sketch, generate, adjust the sketch, regenerate. The infinite canvas made it easy to explore multiple variations side by side.