K-Means Clustering on the Handwritten Digits Data using Scikit Learn in Python
What is k-means Clustering?
A well-liked unsupervised machine learning approach called K-means clustering is used to cluster or organize data points into discrete groups based on their similarity. The sum of squared distances between the data points and their cluster centroids, or the within-cluster variance, is what the method seeks to minimize.
Use of k-means:
K-means clustering divides data points into discrete groups based on similarity and is typically used for unsupervised machine-learning tasks. Here are some typical K-means clustering use cases and applications:
- K-means clustering may divide clients into categories based on their demographics, purchasing patterns, or other characteristics. Businesses may use this information to customize their marketing plans and offerings for each consumer category.
- K-means clustering may compress photos by minimizing the number of colors used. The algorithm produces a compressed image with less storage space, which determines representative colors and assigns each pixel to the nearest representation.
- Anomaly Detection: By locating typical clusters in a dataset, K-means clustering can assist in identifying anomalies or outliers that deviate from the norm. These abnormalities might result from fraud, system flaws, or other inconsistencies in various fields.
- Information retrieval, topic modeling, and document organization are all made more accessible by K-means clustering, which groups documents based on their content or similarity.
- Market Basket Analysis: Using K-means clustering, connections or trends in client purchase data may be found. It can reveal product categories regularly purchased collectively, enabling targeted cross-selling and recommendation systems.
- K-means clustering may divide photos into several parts or areas based on how closely their colors or textures match. Computer vision, object identification, and image processing jobs can all benefit from this method.
- Gene Expression Analysis: K-means clustering may analyze gene expression data to find gene groups that express similarly, which can help understand gene function, classify diseases, and find new drugs.
- K-means clustering may be used in social network analysis to classify users or nodes according to their connections, interests, or behavior. Detecting influencers, locating communities, and customizing suggestions can all be aided by this.
These are just a few examples of the many fields in which K-means clustering may be used. It is a well-liked option for clustering assignments because of its simplicity, interpretability, and efficiency.
Here's an example of how you can perform K-means clustering on the handwritten digits dataset using Scikit-Learn in Python:
from sklearn.datasets import load_digits
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.preprocessing import scale
from sklearn.decomposition import PCA
from time import time
import numpy as np
from sklearn import metrics
# Load the digits dataset
digits = load_digits()
# Extract the data and target labels
data = digits.data
target = digits.target
print("First handwritten digit data:", data[0])
sample_digit = data[0].reshape(8, 8)
plt.imshow(sample_digit)
plt.title("Digit image")
plt.show()
# Scale the data
scaled_data = scale(data)
k = 10
kmeans = KMeans(init="random", n_clusters=k, n_init=10, random_state=0)
def evaluate_kmeans(estimator, name, data):
initial_time = time()
estimator.fit(data)
print("K-means initialization:", name)
print("Time taken: {:.3f}".format(time() - initial_time))
print("Homogeneity: {:.3f}".format(metrics.homogeneity_score(target, estimator.labels_)))
print("Completeness: {:.3f}".format(metrics.completeness_score(target, estimator.labels_)))
print("V_measure: {:.3f}".format(metrics.v_measure_score(target, estimator.labels_)))
print("Adjusted random: {:.3f}".format(metrics.adjusted_rand_score(target, estimator.labels_)))
print("Adjusted mutual info: {:.3f}".format(metrics.adjusted_mutual_info_score(target, estimator.labels_)))
print("Silhouette: {:.3f}".format(metrics.silhouette_score(data, estimator.labels_, metric='euclidean', sample_size=300)))
kmeans = KMeans(init="random", n_clusters=k, n_init=10, random_state=0)
evaluate_kmeans(estimator=kmeans, name="random", data=data)
kmeans = KMeans(init="k-means++", n_clusters=k, n_init=10, random_state=0)
evaluate_kmeans(estimator=kmeans, name="k-means++", data=data)
# Reduce the dataset using PCA
pca = PCA(n_components=2)
reduced_data = pca.fit_transform(data)
kmeans.fit(reduced_data)
# Calculate the centroids
centroids = kmeans.cluster_centers_
labels = kmeans.fit_predict(reduced_data)
unique_labels = np.unique(labels)
# Plot the clusters with labels
plt.figure(figsize=(8, 8))
for i in unique_labels:
plt.scatter(reduced_data[labels == i, 0], reduced_data[labels == i, 1], label='Cluster {}'.format(i))
plt.scatter(centroids[:, 0], centroids[:, 1], marker='x', s=169, linewidths=3, color='k', zorder=10)
plt.legend()
plt.title('K-means Clustering of Handwritten Digits')
plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')
plt.show()
Output:
First handwritten digit data: [ 0. 0. 5. 13. 9. 1. 0. 0. 0. 0. 13. 15. 10. 15. 5. 0. 0. 3. 15. 2. 0. 11. 8. 0. 0. 4. 12. 0. 0. 8. 8. 0. 0. 5. 8. 0. 0. 9. 8. 0. 0. 4. 11. 0. 1. 12. 7. 0. 0. 2. 14. 5. 10. 12. 0. 0. 0. 0. 6. 13. 10. 0. 0. 0.]
K-means initialization: random Time taken: 2.442 Homogeneity: 0.739 Completeness: 0.748 V_measure: 0.744 Adjusted random: 0.666 Adjusted mutual info: 0.741 Silhouette: 0.171 K-means initialization: k-means++ Time taken: 1.755 Homogeneity: 0.742 Completeness: 0.751 V_measure: 0.747 Adjusted random: 0.669 Adjusted mutual info: 0.744 Silhouette: 0.186
The code carries out k-means clustering on a collection of handwritten digits. The first digit's picture is shown after loading the dataset for numbers. The data is then scaled to make its characteristics consistent. The code defines a K-means clustering model with a predetermined number of clusters.
The performance of the K-means model is then evaluated using a function named evaluate_kmeans(). It determines the number of assessment criteria, including silhouette score, adjusted mutual information, adjusted rand index, homogeneity, completeness, and V-measure. The model, the name of the initialization technique, and the data are all inputs to the function.
The "random" and "k-means++" initialization techniques are used to assess the K-means model twice. The evaluation findings, which offer perceptions of the caliber of the clustering outcomes, are printed.
The data's dimensionality is then reduced to 2 dimensions using PCA (Principal Component Analysis). The centroids and anticipated labels for the data points are generated when the reduced data is fitted to the K-means model.
The clusters are then displayed in a scatter plot with a distinct color for each cluster to symbolize it. An 'x' denotes the centroids. The clusters created by the K-means algorithm are visualized in the plot.