Matrix Multiplication Intuition
See matrix multiplication as many dot products and as a transformation of space
A taste of a lesson
I have a 32 x 784 input batch and a weight matrix of 128 x 784. How do I multiply them?
Check shapes first. 32 x 784 times 128 x 784 does not work, because the inner dimensions, 784 and 128, do not match. Transpose the weights: 784 x 128. Now 32 x 784 times 784 x 128 gives 32 x 128: one row per example, one column per output neuron. Each entry is the dot product of one example's 784 inputs with one neuron's 784 weights. Then add the bias to every row. Quick check: how many multiplications does that product take?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Check whether two matrices can be multiplied using their shapes
- Multiply small matrices by hand as rows times columns
- Interpret a matrix as a transformation of space
- Explain why the order of multiplication matters
- Describe a neural network layer and batch processing as matrix products
Lesson plan
- 1 Shapes first Use the shape rule to check and predict products. Start
- 2 Rows times columns Compute products as dot products of rows and columns. Start
- 3 Matrices move space See matrices as stretches, rotations and shears. Start
- 4 Composition and order Understand combined transformations and why order matters. Start
- 5 A neural network layer Read a layer as a matrix product plus bias and activation. Start
- 6 Batches and cost See why batching and hardware matter for AI. Start
Try asking
About this tutor
A beginner friendly tutor for learners who know vectors and want to understand the operation at the heart of neural networks. You will multiply small matrices by hand, learn the shape rule that tells you whether a product is possible, and see each output entry as a dot product. Then you will view matrices as transformations that stretch, rotate and combine, and understand why order matters. Final lessons show how a neural network layer is a matrix multiplication plus a bias, and why processing data in batches is just multiplying bigger matrices.
Reviews
4.7
3 ratingsSample
- Sunita G.Sample
Clear and careful. I finally understand why order matters in matrix products.
- Andre M.Sample
The rotation examples made the transformation view click. A few more practice problems would help.
- Ewa K.Sample
Writing shapes first is such a simple habit and it fixed most of my errors. The neural network layer lesson connected everything.
About the teacher
Linear algebra for AI, with geometry first and notation second
9 tutors 338 lessons taught Sample
I teach the linear algebra behind modern AI: vectors, matrices, similarity, eigenvectors and the methods built on them, such as PCA, clustering and recommender systems. I trained in applied mathematics and later worked on search and recommendation features, so I like to connect each idea to something a real system does. My lessons begin with pictures and small numbers you...
See Katarzyna's profile and tutorsMore like this
Other tutors on the same or nearby topics.