This article explains the fundamental mechanisms behind image generation AI (such as text-to-image models) for non-engineers. As generative AI rapidly evolves, the goal is to provide an intuitive understanding of the underlying principles. For a more technical deep-dive, please refer to the foundational paper behind Stable Diffusion:

(arXiv link) High-Resolution Image Synthesis with Latent Diffusion Models

Images as Digital Data

First, let’s look at how computers represent images as data.

An image is an aggregation of pixels. A 1000 x 1000 image consists of 1,000,000 pixels. In standard digital imagery, each pixel is stored as an 8-bit integer (a number from 0 to 255), meaning any digital image is simply a collection of numerical values:

The above is a grayscale example. For standard RGB color images, each pixel contains 3 separate 8-bit values (Red, Green, Blue), tripling the data volume.

What Humans Recognize as “Art” or “Meaningful Images”

In a 1000x1000 pixel space (1 million pixels), how many unique images are possible? Since each pixel has 256 states, the total number of possible images is:

$$ N_{all} = 256^{1000000} $$

This number is unfathomably huge.
Now, even without deep learning, we could theoretically build a rudimentary “image generator”: simply pick a random number between 0 and 255 for all 1,000,000 pixels.

However, almost anyone looking at these images will perceive them as meaningless noise (static). Every possible photograph of cats, cars, or paintings mathematically exists within that gigantic space, yet purely random sampling will virtually never produce one. Why?

The “Distribution” of Images

Randomly generated images are meaningless because the set of images humans recognize as meaningful pictures is an infinitesimal subset of all mathematically possible pixel combinations. The concentration of desired data within this vast space is called a probability distribution.

*Even abstract art or brush strokes occupy a very specific, structured distribution compared to pure random white noise.

Visualizing the Image Distribution

To visualize this concept, let’s simplify from 1 million pixels down to a 1x2 image (just 2 pixels!). With 2 pixels, the entire universe of images fits in a 256 x 256 coordinate grid.

Suppose we invite 300 artists to submit entries for a “2-Pixel Painting Contest”. Because each image is 2 pixels, we can plot all 300 artworks in 2D space:

Even though each artist chose different pixel values, there is a clear pattern: the artworks cluster into three distinct regions in the space. This clustering is the distribution of artistic images.

Training Generative AI

Image generation AI is trained on vast datasets of images. At its core, training is the process of learning this probability distribution.

Specifically, image generation consists of two main steps:

1. Training (Learning): Mathematically estimating the distribution of valid images.
2. Generation (Inference): Sampling new images from high-probability regions of the learned distribution.

The better the AI approximates the true data distribution during training, the more realistic and natural the generated images will look.

Modern deep learning architectures—including Diffusion Models (like Stable Diffusion) and Transformers—are essentially powerful mathematical engines for learning and sampling from complex high-dimensional distributions.

Hands-On: Training and Generating

Let’s train an AI on our “2-Pixel Painting Contest” dataset.

For this demonstration, we use a Gaussian Mixture Model (GMM)¹. While GMM is classical statistical learning rather than deep neural networks, it effectively models multi-modal distributions and generates new samples.

Fitting the model to the 300 paintings gives us the learned probability distribution below. The darker purple regions represent areas where human paintings are most likely to exist:

Generating a new painting simply means picking a random sample from the high-probability density regions. This produces a brand-new image that is aesthetically consistent with the training distribution without being an exact duplicate of any single training image.

Generating Images with a Matching “Style”

A popular application of image generation is taking a reference image and generating new artworks with a matching style. How is this achieved?

Learning high-level concepts requires large datasets; an AI cannot learn an entire style representation from a single reference image alone. Instead, it relies on a pre-trained model.

Suppose in our 2-pixel model that Pixel 1 corresponds to “Artistic Style” and Pixel 2 corresponds to “Scenery / Subject”².

If we extract the “Style” coordinate from an unseen reference artwork, we can fix the Style value and sample along the remaining distribution dimensions to generate images with different “Scenery” while preserving the exact same “Style”.

That is essentially how modern generative AI paints new pictures!