Virtual Sorting System with Vision Algorithms

A virtual sorting station integrated with vision algorithms.

Project Information

This project focused on the development of a virtual environment integrated with a Programmable Logic Controller (PLC) for automated system control. Advanced computer vision algorithms were implemented to classify products and detect defects in real time, to sort the products correctly. The solution enabled seamless interaction between vision-based inspection and automated control, enhancing both accuracy and efficiency in product handling.

Although the project was developed within a virtual environment, the technologies and systems implemented are fully transferable to physical hardware. The PLC logic was programmed in CODESYS using Sequential Function Chart (SFC) and Ladder Diagram (LD) languages, and executed on a CODESYS virtual controller. Robot integration was achieved by interfacing directly with joint axis data from the ABB RobotStudio virtual controller, where the robot was programmed using ABB RAPID. Communication and operational control between system components were established through a simulated I/O configuration, replicating real-world signal exchange. Communication between the Python-based vision algorithms and the PLC is established via the PLC’s built-in OPC UA server, with the script accessing data through nodes defined in the Global Variable List (GVL).

Showcasing how the system communicates between four different softwares to run the process.

Overview of product input side, showing the control buttons to start the process and handle raised alarms.

The virtual environment within ABB RobotStudio. The Simumatik geography model is imported to accurately define targets.

The data collecting substation, sampling images from several angles automatically.

Custom built robot gripper, mostly used in visual fashion.

Project Details

Project Name	Virtual Sorting System with Vision Algorithms
Purpose	Integrating vision algorithm like Convolutional Neural Networks to sort products in a virtual environment.
Tech Stack	Python TensorFlow / Keras OpenCV Simumatik CodeSYS ABB RobotStudio
Main Features	Product variants classifying by a Convolutional Neural Network Defect detection using color detection algorithm Object position detection, to aid equipment such as robots to determine pick targets PLC Alarm handling, if a product is removed during transit Four-Light beacon to indicate operational status Modular structure of PLC program, making expansion of additional variants easy Energy-efficient operation of conveyors, using only the necessary ones at all times Substation for image data collection, generating images from several angles of related products Built-in image-labeling script, to make labelling of image data fast and easy Separate tasks for Main sorting station and data collecting substation, allowing for concurrent operation
Development Time	~1 Month
Project Type	Course Assignment Project Solo Project
Status	Complete