Using GPT-4o for Customs Classification

The new ChatGPT model is out so I thought this would be a good excuse to try running through some test descriptions and classifications through the model to see how it would go. This new 4o model will be available to free users (unlike the current GPT-4 model) and allows for multiple forms of input including text, images, and voice. The OpenAI YouTube channel shows various demos of how it can be used and the conversational ability looks quite impressive. That said for the purposes of classification the combination of multiple inputs could provide some improvements over the vision models and GPT-4 models I have tested before with the added benefit that 4o should be quicker and cheaper.

Testing the API itself

Before testing the website I modified previous code I had for querying the API and updated the interface a bit then tested with a suitcase picture and description from this link:

The response from 4o was around 2-4 seconds and the model was able to take in both an image and text, using both to classify. Several calls to the model provided consistent responses and the code provided was a valid one going by the current UK customs tariff. Cost wise 4o claims to be half the price of GPT4-Turbo and even less than GPT-4 and the older vision models. This was reflected when I checked my usage and for each classification the cost to run the test was less than 1 cent.

I also tried a pair of shoes available on Amazon here:

Again a legitimate code, quick response and low cost. The model provided explanations as to why it made its decision which can all be outputted to a file if wanted.

Now to try something else I asked a custom query and asked if the model thought the picture was representative of the description:

The model was able to recognise that the description was a good one or at least resembled the image provided. However, I’m always suspicious of whether this is really checking so let’s see how it goes when I replace the image link with one of a car:

The model was able to say that the image was not appropriate and was able to say the image was actually that of a blue car. This kind of recognition between descriptions and images could make identifying inappropriately used images much easier regardless of classification.

How about writing a description from an image? I asked the model this just from the shoe image provided earlier and got this response:

The product in the image is a casual men's shoe. It features a textile upper with a lace-up design and eyelets for the laces. The shoe has a padded interior lining for added comfort. The design combines both canvas and suede materials, with the heel area reinforced by suede for durability. The sole is constructed from a lightweight, contoured, rubber-like material, likely designed to provide excellent cushioning and support. The color scheme is predominantly earthy and neutral shades, making it versatile and suitable for casual wear.

Not a bad description really and also could be used to supplement existing descriptions.

Website model test

To see how 4o does with classification I tried one of the suitcase images and added the description as well from this link resulting in this prompt:

As a UK customs classification specialist can you classify this image to a 10 digit commodity code? This is the description of the goods to help With a highly durable but lightweight Polyester construction, this Cabin Suitcase has been designed to withstand the rigours of travelling. A well-presented interior design enables an organised and stress-free journey, while the exterior also features additional pockets. The addition of a TSA lock further enhances the appeal and aids the security of this Lightweight Cabin Luggage option. Features: 100% Polyester construction 4 x fully rotating spinner wheels Double tube push-button telescopic wheel handle 2 front pockets Top carry handle TSA combination lock 2 bright colour choices Super lightweight – weighs just 2.6kg Excellent packing capacity of 38 litres Cabin sized suitcase* Matching Luggage Set also available *Please check with your chosen airline prior to flying to ensure this Cabin Case conforms to their cabin size restrictions

Despite a quick and crude prompt the response from 4o was:

Based on the provided description of the goods and the image, the item is a soft-sided suitcase made of polyester, designed for use as cabin luggage.

For customs classification purposes, you would typically refer to the Harmonized System (HS) and the specific commodity code that matches the description of the item.

In the UK, the 10-digit commodity code for a suitcase of this type would fall under Chapter 42 of the UK Trade Tariff, specifically under the heading for "Trunks, suitcases, and similar containers."

Here is the classification breakdown:

- **Chapter 42**: Articles of leather; saddlery and harness; travel goods, handbags, and similar containers; articles of animal gut (other than silkworm gut)

- **Heading 4202**: Trunks, suitcases, vanity cases, executive-cases, briefcases, school satchels and similar containers, travelling-bags, insulated food or beverage bags, toiletry bags, rucksacks, handbags, shopping-bags, wallets, purses, map-cases, cigarette-cases, tobacco-pouches, tool bags, sports bags, bottle-cases, jewellery boxes, powder-cases, cutlery cases and similar containers, of leather or of composition leather, of plastic sheeting, of textile materials, of vulcanised fibre or of paperboard, or wholly or mainly covered with such materials or with paper

- **Subheading 4202 12**: Trunks, suitcases, vanity cases, executive-cases, briefcases, school satchels and similar containers, with outer surface of plastic sheeting or of textile materials

- **4202 12 50**: With outer surface of textile materials

Thus, the specific 10-digit commodity code for this item would likely be:

**4202 12 50 90** - Other

Therefore, the full UK commodity code for a polyester cabin suitcase with these features would be **4202 12 50 90**.

This classification considers the materials (polyester), the type of item (suitcase), and its intended use (travel/cabin luggage). Always ensure to verify the latest updates to the tariff schedules and any specific national regulations or requirements.

Problem is though this commodity code doesn’t exist and testing a number of times and trying to correct the model didn’t help either. Retesting using the prompt I have automated in the API python script I use above though was better and gave this result:

Commodity Code: 4202 12 91 90

Explanation:

The product in the image is a cabin suitcase made of polyester, which falls under the category of "Trunks, suitcases, vanity cases, executive-cases, briefcases, school satchels and similar containers; travelling bags, insulated food or beverage bags, toilet bags, rucksacks, handbags, shopping bags, wallets, purses, map-cases, cigarette-cases, tobacco-pouches, tool bags, sports bags, bottle-cases, jewellery boxes, powder-boxes, cutlery cases and similar containers; articles of a kind normally carried in the pocket or in the handbag, with outer surface of plastic sheeting or of textile materials."

Heading 4202: This heading includes various types of containers such as trunks, suitcases, and similar containers.

Subheading 4202 12: This specific subheading is for trunks, suitcases, and similar containers with an outer surface of plastic sheeting or textile materials.

Subheading 4202 12 91: Further narrows down to containers with an outer surface of textile materials.

Taric Code 4202 12 91 90: This specific code is for other suitcases and similar containers with an outer surface of textile materials, not elsewhere specified.

Considering the detailed features and materials mentioned, this classification is accurate. The TSA lock, lightweight design, and specific use as cabin luggage all support this classification under the given code.

Much better with a classification that does exist as well. Prompting is critical to making sure the model provides a good result and something that should not be underestimated when developing any kind of LLM system.

Using the website over an API has downsides though as automation is more difficult and long prompts take time to write. Automating the prompt and query via API enables batching and long prompts can be easily repeated.

Is it better?

Well it is cheaper and quicker than the previous models I have tested and the results appear broadly accurate. I am not seeing improvements over the existing GPT-4 model though that I have tried previously. For simpler classifications I do think that 4o is in a place where it could at the least act as a checker for existing classifications businesses have in place. Running lists of descriptions, images (where available) and current classifications through would take a minimal amount of time and cost and I am sure many businesses are dabbling with this already.

The 4o model isn’t fully released yet at the time of this post but new features should roll out in the coming weeks. In addition the Claude model by Anthropic has recently been opened up to the EU so I can see how this alternative does against ChatGPT.

Code used to test the API

If you want to test the API using code pictured above the code is below. I used PyQT5 for the UI over tkinter this time. You will need your own API key which you can get from OpenAPI for a minimal cost. Note the below code is far from perfect but good enough to do some tests.

import sys
from PyQt5.QtWidgets import (QApplication, QWidget, QLabel, QTextEdit, QPushButton, QVBoxLayout, QComboBox, QFormLayout, QGroupBox, QGridLayout, QHBoxLayout, QCheckBox)
from PyQt5.QtCore import Qt, QMimeData
from openai import OpenAI
import os

# Set OpenAI API Key
os.environ['OPENAI_API_KEY'] = 'YOUR API KEY'
client = OpenAI()

class PlainTextEdit(QTextEdit):
    def insertFromMimeData(self, source: QMimeData):
        if source.hasText():
            self.insertPlainText(source.text())

class ProductClassifierApp(QWidget):
    def __init__(self):
        super().__init__()
        self.initUI()

    def initUI(self):
        self.setWindowTitle('Product Classifier')
        self.setGeometry(100, 100, 1000, 600)  # Adjusted window size for better layout

        # Main Layout
        main_layout = QVBoxLayout()
        self.setLayout(main_layout)

        # Grid Layout for Form and Side Panel
        grid_layout = QGridLayout()
        main_layout.addLayout(grid_layout)

        # Form Layout for Product Details
        form_layout = QFormLayout()
        form_group_box = QGroupBox("Product Details")
        form_group_box.setLayout(form_layout)
        
        # Side Panel Layout for Import Country and GPT Model
        side_layout = QVBoxLayout()
        side_panel = QGroupBox("Settings")
        side_panel.setLayout(side_layout)

        # Adding Form and Side Panel to Grid Layout
        grid_layout.addWidget(form_group_box, 0, 0, 1, 2)  # Span 2 columns for wider form
        grid_layout.addWidget(side_panel, 0, 2, 1, 1)

        # Product Description
        self.description_entry = PlainTextEdit()
        self.description_entry.setPlaceholderText("Enter the product description here")
        form_layout.addRow('Description:', self.description_entry)

        # Materials
        self.materials_entry = PlainTextEdit()
        self.materials_entry.setPlaceholderText("Enter the materials information here")
        form_layout.addRow('Materials:', self.materials_entry)

        # Other Information
        self.other_info_entry = PlainTextEdit()
        self.other_info_entry.setPlaceholderText("Enter other relevant information here")
        form_layout.addRow('Other Information:', self.other_info_entry)

        # Image Link (initially hidden)
        self.img_label = QLabel('Image Link:')
        self.img_box = PlainTextEdit()
        self.img_box.setPlaceholderText("Enter the image URL here")
        form_layout.addRow(self.img_label, self.img_box)

        # Classify Button (moved out of settings)
        self.classify_btn = QPushButton('Classify')
        self.classify_btn.clicked.connect(self.classifyProduct)
        form_layout.addRow(self.classify_btn)

        # Drop-downs for Import Country and GPT Model in Side Panel
        self.import_country = QComboBox()
        self.import_country.addItems(["UK", "USA", "European Union"])
        side_layout.addWidget(QLabel('Import Country:'))
        side_layout.addWidget(self.import_country)

        self.gpt_model = QComboBox()
        self.gpt_model.addItems(["gpt-4o", "gpt-4-0125-preview", "gpt-4-vision-preview"])
        self.gpt_model.currentTextChanged.connect(self.update_ui_elements)
        side_layout.addWidget(QLabel('GPT Model:'))
        side_layout.addWidget(self.gpt_model)

        # Custom Query Checkbox and Text Box
        self.custom_query_checkbox = QCheckBox("Enable Custom Query")
        self.custom_query_checkbox.stateChanged.connect(self.toggle_custom_query)
        side_layout.addWidget(self.custom_query_checkbox)

        self.custom_query_text = PlainTextEdit()
        self.custom_query_text.setPlaceholderText("Enter your custom query here")
        self.custom_query_text.setEnabled(False)  # Initially disabled
        side_layout.addWidget(self.custom_query_text)

        # Output Box
        self.output_box = PlainTextEdit()
        self.output_box.setReadOnly(True)
        self.output_box.setPlaceholderText("API response will be displayed here")
        main_layout.addWidget(QLabel('Output:'))
        main_layout.addWidget(self.output_box)

    def toggle_custom_query(self):
        if self.custom_query_checkbox.isChecked():
            self.custom_query_text.setEnabled(True)
        else:
            self.custom_query_text.setEnabled(False)

    def update_ui_elements(self):
        model = self.gpt_model.currentText()
        self.img_label.show()
        self.img_box.show()

    def classifyProduct(self):
        prodDescription = self.description_entry.toPlainText()
        prodMaterial = self.materials_entry.toPlainText()
        prodOther = self.other_info_entry.toPlainText()
        model = self.gpt_model.currentText()
        importCountry = self.import_country.currentText()
        imgLink = self.img_box.toPlainText()
        custom_query = self.custom_query_text.toPlainText()

        if custom_query:
            message_content = custom_query
        else:
            message_content = f"""The description of the product is {prodDescription}. The materials are {prodMaterial}, 
            other information is {prodOther}. You must classify to a 10 digit level"""

        if model in ["gpt-4-vision-preview", "gpt-4o"]:
            testOutput = self.vision_api_call(prodDescription, prodMaterial, prodOther, model, importCountry, imgLink, message_content)
        else:
            testOutput = self.api_call(prodDescription, prodMaterial, prodOther, model, importCountry, message_content)

        self.output_box.setPlainText(testOutput)

    def api_call(self, prodDescription, prodMaterial, prodOther, model, importCountry, message_content):
        try:
            response = client.chat.completions.create(
                model=model,
                messages=[
                    {"role": "system", "content": f"""
                    You are the {importCountry} Customs Code Helper and are equipped with the ability to classify products
                    based on a text description, material information and other details. This bot remains formal and professional, 
                    providing clear and concise 10-digit commodity codes for {importCountry} customs. The bot will present the code, 
                    followed by a well-reasoned explanation for its selection, helping users understand the classification. The bot 
                    maintains a formal tone, avoiding colloquial language and humor. The bot's priority remains accuracy and relevance 
                    in classifications, structured to first deliver the commodity code, then explain the reasoning behind it, aiding 
                    users in their understanding of {importCountry} customs procedures. It's important to note that all recommendations 
                    provided by this bot are based on available information and are intended as guidance only. Users are advised to 
                    conduct proper research and consult official sources or professionals before making any customs declaration decisions."""},
                    {"role": "user", "content": message_content}
                ]
            )
            generated_text = response.choices[0].message.content.strip()
        except Exception as e:
            generated_text = f"Error: {str(e)}"
        return generated_text

    def vision_api_call(self, prodDescription, prodMaterial, prodOther, model, importCountry, imgLink, message_content):
        try:
            response = client.chat.completions.create(
                model=model,
                messages=[
                    {"role": "system", "content": f"""
                    You are the {importCountry} Customs Code Helper and are equipped with the ability to classify products
                    based on a text description, material information, image, and other details. This bot remains formal and professional, 
                    providing clear and concise 10-digit commodity codes for {importCountry} customs. The bot will present the code, 
                    followed by a well-reasoned explanation for its selection, helping users understand the classification. The bot 
                    maintains a formal tone, avoiding colloquial language and humor. The bot's priority remains accuracy and relevance 
                    in classifications, structured to first deliver the commodity code, then explain the reasoning behind it, aiding 
                    users in their understanding of {importCountry} customs procedures. It's important to note that all recommendations 
                    provided by this bot are based on available information and are intended as guidance only. Users are advised to 
                    conduct proper research and consult official sources or professionals before making any customs declaration decisions."""},
                    {
                        "role": "user",
                        "content": [
                            {"type": "text", "text": message_content},
                            {
                                "type": "image_url",
                                "image_url": {
                                    "url": imgLink,
                                },
                            },
                        ],
                    }
                ],
                max_tokens=300,
            )
            generated_text = response.choices[0].message.content.strip()
        except Exception as e:
            generated_text = f"Error: {str(e)}"
        return generated_text

if __name__ == '__main__':
    app = QApplication(sys.argv)
    ex = ProductClassifierApp()
    ex.show()
    sys.exit(app.exec_())