Explain Information gain

874    Asked by AshishSinha in Data Science , Asked on Nov 4, 2019
Answered by Ashish Sinha

Information gain is a measure of reduction in entropy caused by partitioning the examples according to a given attribute.

The idea is to start with mixed classes and to continue partitioning until each node reaches its observations of purest class. At every stage, the variable with maximum information gain is chosen in a greedy manner. Information gain works on the below formula.

Information Gain = Entropy of Parent - sum (weighted % * Entropy of Child)

Weighted % = Number of observations in particular child/sum (observations in all child nodes)

Information gain works on the below formula




Your Answer

Interviews

Parent Categories