Recognition of Hand-Drawn Hydrocarbon Structure Formulas Using Anchor-Free Detector

The recognition of hand-drawn chemical molecular formulas is crucial for applications such as electronic note-taking and automated grading. Despite the challenges posed by stylistic variations in hand-drawn chemical structure diagrams, we introduce a novel recognition algorithm for hand-drawn hydrocarbon molecular formulas using anchor-free object detection methods. First, we employ an anchor-free detector based on irregular quadrilaterals to identify all potential chemical bonds in input images. By analyzing the collision relationships between these bonds, we then reconstruct all unspecified carbon atoms and assemble them into an adjacency matrix. Finally, we use the RDKit to convert the adjacency matrix into a SMILES string. Notably, our method does not rely on the SMILES string used during training, thereby enabling it to recognize previously unseen hydrocarbons. To verify the effectiveness of the algorithm, we collected a dataset containing 4,217 hand-drawn hydrocarbon molecular structures. Using RepVGG-A0 at a $$512\,\times \,512$$ resolution, our algorithm achieved a recognition accuracy of 85.86%.