It is highly undetectable of steganographers who avoid utilizing public sources of images or steganographic schemes. In such scenario where steganalysers have little priori knowledge about steganographers, clustering is more practical. Ker proposed a MMD-based clustering scheme to distinguish steganographers from innocent actors after comparisons in various configurations and indexes. MMD merely considers the distance between centers of samples from two classes, but ignores the fact that aggregation how samples gather around their centers does affect the separability. Hence, its accuracy needs improvement. To increase the detecting rate further, we propose a clustering based steganalytic scheme using kernel Fisher discriminant indexes (KFDI) as the dissimilarities of samples. We firstly extract the calibration features PEV274 and have them normalized. Then, we calculate the KFD indexes between samples to form the distance matrix. Finally, hierarchical clustering is proceeded with bottom-up iteration where we used the center of gravity as the center for the new gathered clusters. KFDI considers not only between-class variances that maximum mean discrepancy concentrates on, but also within-class variance that affects the aggregation between classes. Experimental results show that our scheme obtains a high increase in accuracy under low embedding rates, about 30% at most, but a little decrease of no more than 5% under high embedding rates. The key contribution of this paper is to propose a more reasonable indicators and steganalytic method based on the KFDI, and we raised the average accuracy of existing methods.