Port Automation with AI based OCR for Adani

Adani, India’s largest ports and logistics company

Adani Ports and Special Economic Zone Limited (APSEZ) is India's largest private multi-port operator. Their largest cargo port in Asia, at Mundra, handles over 10 terminals and processes 100 millions tonnes of cargo annually. APSEZ knew that in order to improve process efficiency and scale port operations, it would need to introduce machine learning based automation for freight management at the port.

Adani’s major challenge in freight container management

It is vital for the port authorities to capture information on the shipping container such as container number, ISO number, seals, license late of the truck carrying container, any hazardous placards or any damages to the container. The container serial number is also used by shippers, carriers, custom, and consignees to identify the cargo content and track the container flow.

It was essential for APSEZ to capture this information accurately because:

• To adhere to regulatory compliance e.g. proper placement of seals
• To determine whether the freight container should be allowed to enter into the port e.g. a heavily damaged container might cause accidents leading to halted operations and economic liabilities to the port
• To determine the destination of the shipping container inside the court after feeding this information into the port operating system and hence directing the truck accordingly

The container serial number is shown in six different positions of the container so that the users can have convenient access to identify the serial number. Currently, the most serial number of the container is identified manually in most container terminals. In another word, it is read by the container inspector without resorting to any automation gadget. Misread number could happen due to the distance, angle, insufficient intensity of light, and poor container position in the yard between the inspector and the container. Reading errors could have also resulted from the inspectors’ oversight. These omissions and mistakes usually occur in lengthy delivery, delays of upto 10 minutes per container and extra expenses. A single mistake can be very costly. APSEZ was looking for a Machine Learning / Computer Vision based solution to automate this process.

Optical Character Recognition for shipping containers

We proposed an Optical Character Recognition (OCR) based Automated Shipping Container Code Recognition system to completely automate their process and remove any human intervention.

Nanonets automated the capture of container images from CCTV cameras, at multiple angles at their port gates at CT@ to streamline the data collection process. We provided a unique solution to Adani that:

• Can leverage the existing hardware to provide an economical solution
• Can be customised according to the KPI Adani wants to track. For example, APSEZ already has a damage severity scale in place and our solution can be customised to give them the damage estimate on the same scale.
• Can be deployed on premise with docker, flexible to work on local private network and can be seamlessly integrated with the port operating system

Our solution deployed at the port was a combination of multiple machine learning models working together in real time. Some of the components of the system were:

• Motion detection to monitor entry of containers
• Deduplication of truck at different cameras at a gate
• Optimal frame classification
• Object Detection and Optical Character Recognition to fingerprint freight containers
• Post processing to verify container details for fraud detection e.g. match number plate captured at the front and back of vehicle, match captured container number at the back, side and top of container
• System integration

OCR API solution overview

Productising a computer vision based solution for Adani was fraught with a number of challenges. Our team’s unique expertise in data science, machine learning, deep learning, research, engineering and architecture enabled us to tackle them in the following ways:

• Sparsity of useful data was a huge challenge. In an hour of video at gained from multiple cameras (in lots of GBs), we could only get data for 20-30 containers. Sparsity of useful data was a huge challenge. To tackle this, we deployed an on-premise smart data capturing solution to capture only relevant info reducing data transfer by over 10,000x.
• We needed to ensure that there is technical feasibility for the flow of a wide variety of data. There was very little data for containers with hazardous placards, damaged containers etc. due to less frequency of such trucks, leading to high bias in the data set. We solved this by augmentation of data.
• Development of a solid technical architecture involved extensive study of data flow in the system and its different parts such as data processors, docker api, motion detector.
• In order to convert the raw data into a format that is easily digestible by machine learning modules, we needed to annotate a huge amount of data. To ensure data privacy, we leveraged our in-house annotation team.
• Once the models were built, we converted the models into docker containers as per architecture so that they can be deployed on the system. In this stage, we also wrote code for all other parts of the system so the solution is modular and can be deployed at scale.
• Since this solution involved on premise deployment, once the testing was done on cloud based deployment, we setup the system with required operating system, appropriate drivers needed for GPU’s and setup dependencies to run the code.
• We re-collected the data where the models made an error and built model building processes to redeploy the new models. The system learns over time and accuracy of the system keeps improving.

Impact of container code automation

The development of advanced container OCR systems is a field where Artificial Intelligence (AI), new technologies such as wireless communications, and traditional image processing can be merged to help achieving higher levels of reliability.

Our AI-based Container Number Recognition System would reduce the need for 45 office workers employed across 15 gates for manual verification and tracking. Additionally, Adani has a real time tracking tool to search and monitor the entry of any container / truck at any of their ports.