Finally, due to the generated skeletons are based on the form of skeleton points and lines so far. Our key innovation is that the attention maps are calculated on both spatial and temporal dimensions in turn, meanwhile, graph convolution is used to strengthen the short-term features of skeletal structure. Then the spatial-temporal graph self-attention utilizes graph topology to capture the intra-frame and inter-frame correlations, respectively. Specifically, according to kinesiology, we first design a novel graph representation to achieve graph features from skeletons. In this paper, we propose a novel method named Spatial-Temporal Graph Transformer (STGT) to deal with this problem.

As a typical sequence to sequence task, the existing methods are mostly to regard the skeletons as a whole sequence, however, those do not take the rich graph information among both joints and edges into consideration. Sign language production aims to automatically generate coordinated sign language videos from spoken language.

Fastcopy 3.92