Living in High-Dimensional Space
Living in High-Dimensional Space
Close your eyes for a moment. Imagine a straight line. Easy. Now imagine a square. Still easy. Now imagine a cube. No problem. But now imagine a thousand-dimensional cube. You can't. Not because it doesn't exist... but because your brain was never built to visualize it. Humans evolved in a three-dimensional world. Length. Width. Height. Everything we touch, build and experience fits within these three dimensions. Artificial Intelligence doesn't have that limitation. To a computer, three dimensions... three hundred dimensions... or three thousand dimensions... are all just collections of numbers. Think about a human face. If I describe it using only Height, Weight, and Age, many people will appear identical. Now add Eye colour, Hair colour, Nose width, Jaw shape, Skin tone, Smile pattern, Distance between the eyes, and hundreds of other characteristics. Every new feature becomes another dimension. The more dimensions we add, the more unique each face becomes. Language works the same way. A sentence is not represented by Length, Width, and Height. It is represented by hundreds or even thousands of numerical features. Every sentence becomes a point inside a gigantic mathematical space. Similar ideas naturally appear close together. Different ideas drift apart. ChatGPT never searches English. It searches this invisible geometric world. This sounds perfect. More dimensions should mean better descriptions. And they often do. But something unexpected happens. As dimensions increase, space begins to behave differently. Imagine standing in an empty football field. Finding your friend is easy. Now imagine searching for that same friend somewhere on Earth. Much harder. Now imagine searching across the entire solar system. The search space exploded. Higher dimensions create the same problem. As the number of dimensions grows, objects spread farther apart. Neighbours become harder to find. Distance begins to lose its intuition. The mathematics itself changes. This phenomenon is called the Curse of Dimensionality. It is one of the greatest challenges in modern Machine Learning. Ironically, the same dimensions that make objects easier to describe... also make them harder to search. This creates a fascinating question. Can we somehow keep the important information... while reducing the number of dimensions? Can we compress a thousand-dimensional world into something much smaller without losing what truly matters? That question led to one of the most beautiful ideas in all of Machine Learning. Principal Component Analysis. Before we learn how PCA compresses dimensions, we need one more mathematical tool. How do we rotate, stretch and transform an entire space without destroying the relationships between its vectors? The answer lies in matrices. And that marks the beginning of the next lesson about Linear Transformations and Matrix Algebra.