Recent progress in pose-estimation methods enables the extraction of sufficiently-precise 3D human skeleton data from ordinary videos, which offers great opportunities for a wide range of applications. However, such spatio-temporal data are typically extracted in the form of a continuous skeleton sequence without any information about semantic segmentation or annotation. To make the extracted data reusable for further processing, there is a need to access them based on their content. In this paper, we introduce a universal retrieval approach that compares any two skeleton sequences based on temporal order and similarities of their underlying segments. The similarity of segments is determined by their content-preserving low-dimensional code representation that is learned using the Variational AutoEncoder principle in an unsupervised way. The quality of the proposed representation is validated in retrieval and classification scenarios; our proposal outperforms the state-of-the-art approaches in effectiveness and reaches speed-ups up to 64x on common skeleton sequence datasets.
SegmentCodeList: unsupervised representation learning for human skeleton data retrieval
Carrara F;Amato G
2023
Abstract
Recent progress in pose-estimation methods enables the extraction of sufficiently-precise 3D human skeleton data from ordinary videos, which offers great opportunities for a wide range of applications. However, such spatio-temporal data are typically extracted in the form of a continuous skeleton sequence without any information about semantic segmentation or annotation. To make the extracted data reusable for further processing, there is a need to access them based on their content. In this paper, we introduce a universal retrieval approach that compares any two skeleton sequences based on temporal order and similarities of their underlying segments. The similarity of segments is determined by their content-preserving low-dimensional code representation that is learned using the Variational AutoEncoder principle in an unsupervised way. The quality of the proposed representation is validated in retrieval and classification scenarios; our proposal outperforms the state-of-the-art approaches in effectiveness and reaches speed-ups up to 64x on common skeleton sequence datasets.File | Dimensione | Formato | |
---|---|---|---|
prod_479562-doc_196802.pdf
Open Access dal 17/03/2024
Descrizione: SegmentCodeList: unsupervised representation learning for human skeleton data retrieval
Tipologia:
Versione Editoriale (PDF)
Dimensione
530.78 kB
Formato
Adobe PDF
|
530.78 kB | Adobe PDF | Visualizza/Apri |
prod_479562-doc_196803.pdf
Open Access dal 17/03/2024
Descrizione: Postprint - SegmentCodeList: unsupervised representation learning for human skeleton data retrieval
Tipologia:
Versione Editoriale (PDF)
Dimensione
714.75 kB
Formato
Adobe PDF
|
714.75 kB | Adobe PDF | Visualizza/Apri |
prod_479562-doc_196804.pdf
Open Access dal 17/03/2024
Descrizione: Preprint - SegmentCodeList: unsupervised representation learning for human skeleton data retrieval
Tipologia:
Versione Editoriale (PDF)
Dimensione
516.96 kB
Formato
Adobe PDF
|
516.96 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.