|
Last updated on Nov 07, 2023
A decision tree uses a supervised machine learning algorithm in regression and classification issues. It uses root nodes and leaf nodes. It relies on using different training models to find the prediction of certain target variables depending on the inputs.
It works well with boolean functions(True or False). The node is the feature, the leaf is the result, and the links show the decision outcome. Most of the decision tree relies on different types of nodes:
Other terms used in the decision tree include:
Want to Become a Master in Machine Learning? Then visit here to Learn Machine Learning Training
There are several ways that ensure that the results are accurate. It uses different algorithms to work with nodes and subnodes that will work with the variables.
Some of the algorithms that work well with decision trees are:
This algorithm begins by first setting the root node as Original S. On every iteration, it loops through all the unused S attributes, and it chooses the attribute with small entropy.
It later partitions it using the selected attribute to have many subsets. It continues recurring on the subset with the consideration of the not selected attributes before. Despite all these, the recursion stops if all the subset elements belong to the same class and no attributes are selected.
There are several solutions this algorithm uses in case of attribute selection issues. Some of these solutions include:
It uses the greedy heuristic method, and the accuracy can increase when you process the data. It checks if there is randomness in the data used in the process. It identifies the outcome of any result. An example is when one tosses a coin, there will be different landings and outcomes.
The formulae of calculating entropy is:
Where S stands for the current dataset, X means set classes in S, and px represents the probability.
Entropy enables users to calculate the expectations of gained information from a specific variable and the value of distributed unknown values. The distribution of a value is known when the entropy is zero. When the entropy is higher at the node, then there is little information about the data, and improvement is needed to improve it.
Sometimes it's referred to as Gini impurity. It uses the CART algorithm to randomly choose an element from a set that can face incorrect labeling issues if randomly labeled by the distribution of labels in a subset.
You calculate it by checking the probability of a certain feature that gets classified wrongly when it gets selected randomly. To get the index, you subtract the sum of all squared probabilities from each class.
The Gini index can be expressed as:
Where Pi is the probability of the elements that get classified from a specific class.
It's a function that uses Entropy. It shows how a certain attribute can separate from other training examples using the classification of the target attribute.
It decides if a feature can split a node or not by looking at the feature with high information gain at the node.
To calculate the Information gain, we first calculate the entropy of the descriptive feature by splitting the dataset using its value minus the entropy of the original dataset. This method works with the C4.5 algorithm.
The formulae for calculating this can be illustrated as:
InformationGain(descriptive feature)=Entropy(original dataset)- Entropy(descriptive feature)
This algorithm is mostly for continuous variables. It measures which feature where nodes get split into child nodes. Variance is mainly for finding a homogenous node(a node that has zero variance)
To calculate variance:
To calculate variance, we use the following formulae:
Sometimes it uses the acronym CHAID meaning CHi-squared Automatic Interaction Detection. It's a statistical method that finds differences between subnodes and the parent nodes. It uses the Chi-square Automatic Interaction Detector tree.
The algorithm is non-parametric, and it works well with large datasets.
You calculate it as:
First, you get the deviation of both the success and failure variables and calculate the chi-square of each node.
Second, you check the sum of both success and failure nodes and later calculate their chi-square of the split.
It has the power to perform several splits at one node, which brings in more accurate results. Its mostly applied in the field of direct marketing to get more clients.
According to Rose Quinlan, the gain ratio is the ratio between information gain and intrinsic information, which normally has a bias towards the multi-value attributes. When you choose the attribute, you consider the one with the largest number by looking at the size and number.
It normally selects the attribute with large values.
The gain ratio can be calculated as:
First, we have to calculate the information gain:
IG(X,a)=H(X)-H(X|a)
Where X is the variable and H(X|a) is the entropy.
Second, we calculate the split information as follows:
Where X is the variable,t is set events, N(ti) is the how many times it appears divided by the number of events N(t)
The gain ratio is calculated as:
There are several types of decision trees depending on the variables. Both categories share some similarities and have some differences. There are two types of decision trees.
Both types use CART analysis(classification and regression tree ).
There are several uses of decision trees in business and other fields. Some of the common applications include:
Companies use historical and competitor data to analyze how their marketing campaigns affect the buying of products and services. It helps them develop better campaigns that will produce more sales and conversions than the competitor.
Companies use decision trees to know the amount of energy needed by each household. You can use variables like the number of household members and one of the equipment each household has, like a refrigerator, and try to determine the result.
Fraud reduces tax collections for many countries and brings losses to many businesses. You use a decision tree to monitor any fraudulent behavior that is suspicious and treat it as fraud.
Engineers use decision trees to find if their rotary machines have any faults in the bearing. It involves using vibrations, signals, and variables during the evaluation process.
Many companies use decision trees with the help of doctors to help identify early symptoms of diseases and ailments. It uses different methodologies and algorithms to find out. Some examples include detecting breast cancer, child diseases, diabetes, e.t.c.It helps in taking preventive measures at the early stages of diseases.
Most companies use decision trees to check the purchasing behavior of their customers. Once you understand their behaviors, they use the trees to recommend new products that meet their behavior, making them love the whole experience.
Decision trees are significant when dealing with machine learning. It uses different algorithms to come out with the best accurate prediction. When working with the trees, it works according to the rules and guidelines you provide.
The majority of the programming languages can create decision trees, and it involves a few steps to achieve the final result. There are a lot of applications of decision trees, but there are methods people use instead of decision trees when working with datasets.
| Batch starts on 8th Dec 2023 |
|
||
| Batch starts on 12th Dec 2023 |
|
||
| Batch starts on 16th Dec 2023 |
|
