PCA in Machine Learning

Machine Learning is one of the most used modern technologies in our world. Machine Learning helps human a lot to do their jobs at ease. PCA is widely used Machine Learning technique. It is an unsupervised Machine Learning technique. In this article we are going to discuss the PCA in details.

What is PCA?

The full form of PCA is Principal Component Analysis. It is a very useful Machine Learning technique. The principle aim of PCA is to find principal components so, that we can describe the data points with the help of the set of the principal components. In this process we will use orthogonal transformation and find out the set of uncorrelated features from co-related features. These new transformed features are called the Principal Components. It is one of the popular tools that are used for exploratory data analysis and predictive modelling. It is a technique to draw strong patterns from the given dataset by reducing the variances. In this PCA algorithm we mainly use the mathematical concepts like Eigen value, co-variance, and variance and Eigen factors.

Terminologies used in Principle component analysis algorithm

1. Orthogonal: This term mainly denotes that the variables which we have in data set are not co-related. So, the variance between them is zero.
2. Dimensionality: This term mainly denotes how many features are present in the data set. For example if we have 100 columns in a data set then the dimensionality will be 100.
3. Correlation: This term mainly denotes the co-relation between two variables or features. In simple word we can say that when one variable changes after changing another then we can say that they are co-related. The value of correlation may be from +1 to -1. Here +1 denotes that they are simply proportional to each other and -1 denotes that they are inversely proportional to each other.
4. Co-variance matrix: This term mainly denotes the matrix which consists of co-variances between pairs of variables.

Steps in PCA algorithm

1. Collection of data set: The first step of this algorithm is to collect the data set from raw data from different sources. We have to divide the data set into two parts. One part will be used as training set and another part will be used as validation set.
2. Representation of the data set in a structure: In this step, we have built a structure of collected data set. The rows will denote items or cases and the columns will denote variable or features.
3. Data standardization: This step will include the process of standardization of the whole data set. In this step we will check the higher and lower variances and after that process the items.
4. Co-variance calculation: In this step, we will produce the co-variance matrix which we have discussed previously. To form this matrix we have to transpose our data matrix and then multiply it with previous one. In this way we can form the matrix.
5. Eigen values and vectors calculation: In this step, we will calculate the Eigen vales and Eigen vectors. We will get them from the co-variance matrix.
6. Sorting of Eigen values: In this step we will sort the Eigen values. The sorting will be in decreasing order it means the higher value will get higher order. Eigen vectors will be sorted ion same manner.
7. Forming new features: At the last we will form the new features or principle components. After selecting principle components we have to delete other components.

Advantages of PCA

1. Principle component analysis algorithm is very easy to implement as you can see above. It includes only some basic computations.
2. Suppose you have a basic data set and you are performing some Machine Learning algorithm on it then the time will high. But when you implement the same Machine Learning algorithm on a data set which has gone through the principle component analysis it means, it has principle components then the algorithm will learn quickly.
3. It is very often seen that if we apply a regression based algorithm on a high dimensional data set then it may occur outfitting. So, to avoid this situation we have to reduce the dimensionality. In above discussion we have already told that principle component analysis reduces the dimensionality.