Jul 4, 2026, 8:44 AM

Found a fun animation I once made for a course.

It shows how two classes that are not linearly separable become linearly separable when using a small non-linear layer.

The task is a generated "two circles" toy dataset. It uses a very small non-linear layer (probably three neurons, cannot remember), a linear projection to 2D (for visualization), and a logistic regression layer for classification.

The animation shows optimization of random weights to a linearly separable solution.

#ml #mlp

Dataset that is not linearly separable (two circles/elipses). An elipse of the red class encloses an elipse of the blue class.

Animation of the two two circles dataset after non-linear projection. With random weights, the classes are not separable, however certain number of optimization steps they become linearly separable. Also some weird oscillations at some point.

Replies