Machine-vision-based reading and sorting devices have been used to measure and classify items. Here, we extend their application to the sorting and assembling of items identified by their geometry and color. In this study, we developed an improved machine vision system that is capable of discerning and categorizing items of distinct geometries and colors and utilizes a computer-controlled robotic system to manipulate and segregate these items. Hence, a machine vision system for an automatic classification process while operating a robotic arm is hereby developed. To obtain positioning information, the proposed system uses cameras that were mounted above the working platform to acquire images. Perspective and quadratic transformations were used to transform the image coordinates of the calibration system to the world coordinates by using a calibration procedure. By these methods, the proposed system can ascertain the two- and three-dimensional coordinates of the objects and automatically perform classification and assembly operations using the data collected from the visual recognition system.