k Nearest Neighbours, Built by Hand
Predict by similarity and learn why distance, scaling and k decide everything
A taste of a lesson
My kNN model on age and salary works badly. Age is 20 to 60 and salary is 20000 to 90000. Could that be it?
Very likely. Two people 30 years apart in age but 500 apart in salary look much closer than two people the same age but 5,000 apart in salary. Distance is almost entirely salary, so age barely counts. Standardise both features: subtract each feature's mean and divide by its standard deviation, using training data only, then apply the same numbers to new points. Try it: with mean age 40 and standard deviation 10, what is the scaled value for age 55?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Classify and predict values by hand using k nearest neighbours
- Compute Euclidean and Manhattan distances
- Explain why feature scaling changes predictions
- Choose k using validation and describe the bias variance trade off
- Describe when kNN becomes slow or unreliable
Lesson plan
- 1 Predicting from neighbours Classify a new point by looking at its closest labelled points. Start
- 2 Measuring distance Compute and compare Euclidean, Manhattan and cosine distance. Start
- 3 Why scaling matters See how unscaled features dominate distance and fix it. Start
- 4 Choosing k Pick k with validation and understand the trade off. Start
- 5 Regression with neighbours Predict numbers by averaging nearby targets. Start
- 6 When kNN struggles Recognise slow prediction and the curse of dimensionality. Start
Try asking
About this tutor
A beginner tutor that uses k nearest neighbours to teach the core ideas of machine learning: similarity, distance, scaling and the trade off between flexible and stable models. You will classify points on a grid by hand, compute distances, see how the choice of k changes predictions, and discover why unscaled features quietly ruin results. Later lessons cover regression with neighbours, weighting, slow prediction on large data and the curse of dimensionality. It is a gentle first algorithm that leaves you with habits you will use for every other model.
Reviews
Students can review a tutor after a paid lesson. Nobody has yet.
About the teacher
Classical machine learning models, worked through on paper before any code
9 tutors 379 lessons taught Sample
I teach the classical machine learning models: regression, trees, ensembles, nearest neighbours, support vector machines and naive Bayes. My background is in applied analytics, where I spent a lot of time explaining to colleagues why a model made a particular prediction, and that shaped how I teach. I like to start with a tiny dataset you can hold in your...
See Kavya's profile and tutorsMore like this
Other tutors on the same or nearby topics.