In this paper, we introduce an automated vision-based system which consists of an image grabbing mechanism and an inspection method for detecting and classifying defects on the surface of leather fabric. The proposed inspection method treats errors like scars, scratches and pinholes. In the defect detection process, several image processing algorithms are employed to extract image features and locate defect’s positions on leather surface. In the defect classification process, these collected features are used to classify the type of defect based on SVM algorithm. Experimental results show that the detected defects are localized according to their real-positions on leather surface. The size and type of detected defect are also accurately determined.