Split it up - Part 2

-

Recently heard in news about ongoing 10th CBSE exams. Then suddenly I recalled  my 10th board exams days . PHEWWW..!!! how dreadful those examination days were. I remember as the exams were approaching near ,school teachers used to increase their teaching speed ,took extra lectures ,sometimes even ate up our sports time , trying to complete syllabus as soon as possible . After finishing the syllabus and taking up many revision sessions by our teachers ,finally we were ready for final battle of our school life that is final exams and fortunately we all scored well. 


Hey wait!! I forgot to mention pre-board exams .After our revision session and when enemy named final exam was just outside our palace ready to attack us soon ,our teacher used to conduct an exam less dreadful than the final exams, commonly known as Pre-board exam.  I always wondered why the heck do they conduct these many exams . I asked my mother who is indeed a very good mathematics teacher as to what  actually does this pre-board exams are meant for. She patiently told me that it is necessary to examine a student's learning before she/he actually gives final exams and check whether a student is able to apply the concept which she/he have learnt during the class and generalize it for every other question rather than just mugging up. If student scores well in the pre-board exam then they are ready for final exam ,if not teacher needs to reteach some concept, give some more practice problems , introspect where the students are facing problems and what can be done. All this problems can be handled and corrected  before final exams because you cannot afford to make mistakes there and hence pre-board exam is a necessary factor to check student's performance on questions similar to final exams .

Ohh !!! is it? Then can we extend this idea to Machine learning as well ??After all there also first training and then testing is performed like that of ours learning phase and exam phase. Answer to this question is a big yesss.

As discussed in the last blog "Split it up -Part 1" ,a dataset is splitted into Train and Test data where Train data is used to train model and test data is used to evaluate performance of the model.But heyy....!! does this guarantee that model will give the same accuracy on future unseen data as  it gave via Test data ?Absolutely not, one cannot guarantee that model will work as good as it worked on Test data. The model also needs to undergo through the same exam that is Pre-board exam to validate how the model is actually working and what changes are required to enhance its performance . In Machine Learning this exam is known as Cross Validation. A dataset is usually splitted into 3 parts namely Train data ,Test data and  Cross Validation data . 
Cross validation is used to make necessary changes in the hyperparameters of the model so that model can perform well on test data which will  absolutely be new data for the model and generalization can be done .  

In a simple manner now we can say that after cross validation, our model is ready to give final exam that is to face test data (completely new questions) .And now result of final exam on test data will give the actual performance of the model . As training/learning is more important and crucial ,more number of datapoints are reserved for Train data (that is more than half of the data reserved for Training) .Test and CV data can have almost similar number of datapoints.


Take away points:
  • Dataset must be splitted into Train,Test and Cross Validation data .
  • Cross validation is necessary to make changes in model before feeding it the test data which is unseen data.
  • Train data must hold maximum number of datapoints than Test and Validation data.
  • Data can be splitted in the ratio Train: CV: Test such as  60:20:20, 70:15:15, 64:16:20 etc. whichever gives good result,  decided by user .

Comments

  1. AnonymousMay 14, 2022 at 9:44 PM

    You made it simple as that!!!

    ReplyDelete
  2. AnonymousMay 14, 2022 at 11:21 PM

    What if I use my entire data for cross validation? Will that be a wrong practice?

    ReplyDelete
    Replies
    1. Mr. Data ScientistMay 15, 2022 at 12:29 AM

      Yes it is absolutely wrong..You need to first train the model via train data ,then and only then you can use cross validation data for hyperparameter tuning and finally evaluate performance of model on Test data

      Delete

Post a Comment

Popular posts from this blog

Covariance and Correlation

-
Image
While dealing with any ML project one spends most of the time in data preprocessing .Data pre-processing is nothing but playing with the features present in the dataset ,data cleaning,data transformation ,removing or imputing missing feature values etc. Feature engineering corresponds to developing  new features based on current information . It may happen that few features are related to each other be it directly or inversely . Consider three cases:  Let's say radius and circumference are features present in the dataset ,so we know that they are directly dependent .Here if radius increases ,perimeter also increases.       2. Let's say speed and time are features present in the dataset ,so we know that they are inversely               proportional  .Here if speed increases ,time required to cover a distance decreases.         3. Let's say height and hair color are features present in the datas...
Read more

Split it up - Part 1

-
Image
M achine  L earning ???? How can a machine even learn??  For a long period of time I always wondered what this fancy term meant and how would it be actually possible ..!! Let's figure it out today. To get an essence of what happens when we say Machine is learning ,why not understand in the first place how we learn. Well..!!! let us take a glance at those days when we learnt mathematics as a kid. I still remember how my maths teacher used to teach us basic addition and subtraction in class .Initially she helped us to count using fingers and then add or subtract those numbers. With a lot of practice and sometimes getting beaten up by her we finally learnt addition /subtraction. Mathematically speaking after lot of practice /training , our brain learnt a function of addition where if we get numbers to add ,it processed in a way to give the sum. Similarly Machine Learning is all about finding that magic function "f" wherein ,we give some input and desire some output . (EXC fo...
Read more

Why activation function is needed in Neural Networks???

-
Image
Today allow me to share with you, the relationship between a spider and an activation function   Activation function is used in Neural network to introduce the non-linearity. Without the proper activation function, neural network will simply be large Linear Model.   Consider the case where you have a data which is linearly separable  Constructing a model to separate yellow and orange data points is very simple. A simple logistic regression will do this job very perfectly  Now consider the case where your data set looks like something shown below What do you think now?? will a simple linear model be able to carve out the hyperplane in such a way to classify both yellow and orange points. I don’t think so. And it is the fact that a simple linear model may not be able to find complex patterns in a data.  So now we go to big brother of linear model -Neural network. Neural Network has this super ability to both carve out linear...
Read more