Despite many years of active research traffic sign detection and recognition is still an open problem. Recognition accuracy and false detections rate are still not sufficient for replacing human operators with automatic recognition system. We are working on several topics in this area:

• high-speed detection
• sign recognition with multi-layer neural networks
• synthetic datasets generation for training

Russian traffic sign images dataset (RTSD)

This dataset is intended for training and testing the algorithms of traffic sign recognition. Frames are obtained from widescreen digital video recorder which captures 5 frames per second. Frame resolution is from 1280×720 to 1920×1080. Frames are captured in different seasons (spring, autumn, winter), time of day (morning, afternoon, evening) and in different weather conditions (rain, snow, bright sun). This dataset surpasses other public traffic sign datasets in number of frames, signs classes, physical signs and images of signs. Sign labeling on frames was spent in two steps. On first step tracks of physical objects were selected on sequential frames. On the second step indistinguishable signs were discarded and every physical sign was assigned a class.

It consists of 205 classes, of which 99 are found only in the test set and are completely absent in the training set, and 106 classes are present in the training set.

Statistics for detection task RTSD dataset:

Statistics for classification task RTSD dataset:

Synthesis of data for the classifier

The problem of recognizing road signs is unsolved for a large number of classes of signs, since there are many rare classes of signs. For systems that recognize road signs, the ability to quickly adapt in the event of a new type of road signs is important. With the emergence of a new class of signs, it is difficult to collect a sufficient number of real training examples. In the presence of synthetically generated images of this sign, the ability to classify a new type of sign can be quickly introduced into the system. High-quality synthetic samples allow us to solve the problem with missing classes and data collection with examples of these signs.

Existing methods for traffic sign processing:

• Synt - this is a simple synthetic, which was obtained by embedding signs on the background and applying a transformation of sign with random parameters to the icon: rotate, shift, contrast change, Gaussian blur, motion blur.
• CGI - samples, which were obtained by rendering three-dimensional models of traffic signs on pillars in real road images.
• CGI-GAN - in this sample, traffic signs are transformed from the CGI collection to better ones using CycleGAN.
• Pasted - this approach is based on embedding synthetic traffic sign on a real traffic sign place. The real traffic sign itself is inpainted. Then embedded sign is being improved with GAN.
• Cycled - this is improvement of Pasted method. Here, the second data stream is added to the training process. The icon of the sign of the same class, which was in a real patch, is embedded. As a result, the entire neural network should ideally get a picture identical to the original one.
• Styled - this approach uses StyleGAN architecture. Neural network generates synthetic traffic sign based on the background image and on a sign icon of required class. The architecture of this approach is in the next figure:

Architecture of generator for processing in Styled approach

Synthesis of data for the detector

The quality of the detectors can also be improved with synthetic signs. In this task, the consistency of the appearance of the sign with the background is especially important. The correct positioning of the new synthetic road sign is also very important.

Existing methods for embedding of traffic signs in images:

• Inpaint – this is a simple synthetic data for the detector, in which an icon of a traffic sign is drawn in the image without any processing.
• Pasted, Cycled, Styled - these methods are initially designed to generate a sign consistent with the input background. Further, the processed new sign is cut out and inserted in the desired place.
• KDE - new places of traffic signs were determined or using kernel density estimation.
• NN-placement - new places of traffic signs were determined or a special neural network.

Real image	Image with additional signs

Current best results

Classification of signs in RTSD dataset:

Detection of signs in RTSD dataset (AUC) with classifier:

Downloads

• RTSD files
• Rare traffic sign classifier
• RTSD-Pasted
• RTSD-Cycled
• RTSD-Styled
• RTSD-NN-placement