In this thesis work, we consider the problem of object clustering and classification from a set of images. The problem being considered here is supervised since we know the class labels of the images. Saliency detection is used as a tool to narrow down the object location in the image. In this thesis work we study a variant of the popular K-means clustering algorithm. Clustering is applied separately for each class, with the bag of salient point feature vectors representing an image assigned to one of a set of cluster “families” where each class is represented by multiple cluster families. A cluster family can be thought of as a sub-class or a class within a class. A supervised variant of this scheme is also proposed whose parameters can be initialized by the (unsupervised) clustering approach. In this case, the “clustering” solution is learned based on a gradient descent procedure, with respect to a “soft” training set measure. We aim to experiment with the number of clusters families used to represent each class and the number of clusters present in each cluster family.