Conventional detection ways of seed vigor cannot meet currently development demands of automation and efficient breeding because of its subjectivity, high-cost and complicated operations. In the study, a visual detection system of vegetable seed vigor was developed, to calculate the seed vigor index (SVI) through image feature extraction, the aim to explore new method for detecting automatically vegetable seed vigor. The detection system was composed of four platforms including image acquisition, image processing, germination index computing and seed vigor index calculating. Vegetable seeds, cucumber, chilli, tomato, and aubergine were chosen to evaluate the capacity of detection system. Via experimental tests between manual and self-developed SVI system, the latter shown higher recognition accuracy through comparative analysis of four vegetable seeds, as well as deviations were respectively 4.32, 4.90, 5.95 and 3.22% and within acceptable limit. Thus, this work provides a reliable reference and research foundation for streamline detection of vegetable seed quality.