(aside image)

CoBaSiL Project

The Content-based video analysis and annotation of Finnish Sign Language (CoBaSiL) research project aims to develop new methods for content-based analysis and annotation of sign language videos. Sign language is an example of a narrow video domain enabling detailed analysis of the semantic contents, while being scientifically interesting for computer vision research due to multiple challenges such as the combination of global and local motion and the presence of occlusions.

The project is a multidisciplinary challenge that calls for expertise in a large variety of scientific fields, and the participants have significant amounts of relevant prior expertise. Aalto University School of Science has a long history in research on content-based image and video retrieval. University of Jyväskylä has conducted research on sign language acquisition focusing in the phonetics and phonology of Finnish Sign Language.

The project is funded by the Academy of Finland and has started on 1.1.2011 and will last until 31.12.2014.

One of the outcomes of the project is the free SLMotion software package for video analysis. It includes many of the algorithms we have used in this project.

Another output of the project is the S-pot benchmark of spotting particular signs within continuous signing that is freely distributed for academic purposes.

Contact: jorma.laaksonen@aalto.fi

People

Links

Publications

2014

Ville Viitaniemi, Matti Karppa, and Jorma Laaksonen. Experiments on Recognising the Handshape in Blobs Extracted from Sign Language Videos. To appear in Proceedings of 22th International Conference on Pattern Recognition (ICPR), Stockholm, Sweden, August 2014.
Info
Marcos Luzardo, Ville Viitaniemi, Matti Karppa, Jorma Laaksonen, and Tommi Jantunen. Estimating Head Pose and State of Facial Elements for Sign Language Video. In 6th Workshop on the Representation and Processing of Sign Languages in connection with the 9th Language Resources and Evaluation Conference (LREC 2014), Reykjavik, Iceland, May 2014.
Info
Ville Viitaniemi, Tommi Jantunen, Leena Savolainen, Matti Karppa, and Jorma Laaksonen. S-pot - a benchmark in spotting signs within continuous signing. In Proceedings of 9th Language Resources and Evaluation Conference (LREC 2014), Reykjavik, Iceland, May 2014.
Info
Matti Karppa, Ville Viitaniemi, Marcos Luzardo, Jorma Laaksonen, and Tommi Jantunen. SLMotion - An extensible sign language oriented video analysis tool. In Proceedings of 9th Language Resources and Evaluation Conference (LREC 2014), Reykjavik, Iceland, May 2014.
Info

2013

Ville Viitaniemi, Matti Karppa, Jorma Laaksonen, and Tommi Jantunen. Detecting Hand-Head Occlusions in Sign Language Video. In Proceedings of the 18th Scandinavian Conference on Image Analysis (SCIA2013), Espoo, Finland, June 2013.
Info
Marcos Luzardo, Matti Karppa, Jorma Laaksonen, and Tommi Jantunen. Head Pose Estimation for Sign Language Video. In Proceedings of the 18th Scandinavian Conference on Image Analysis (SCIA2013), Espoo, Finland, June 2013.
Info
Tommi Jantunen, Ville Viitaniemi, Matti Karppa, and Jorma Laaksonen.The head as a place of articulation: From automated detection to linguistic analysis. In Proceedings of 11th Theoretical Issues in Sign Language Research conference , London, 2013.
Info

2012

Matti Karppa, Tommi Jantunen, Ville Viitaniemi, Jorma Laaksonen, Birgitta Burger, and Danny De Weerdt. Comparing computer vision analysis of signed language video with motion capture recordings. In Proceedings of 8th Language Resources and Evaluation Conference (LREC 2012), Istanbul, Turkey, May 2012. Available online at http://www.lrec-conf.org/proceedings/lrec2012/pdf/321_Paper.pdf.
Info

2011

Matti Karppa, Tommi Jantunen, Markus Koskela, Jorma Laaksonen, and Ville Viitaniemi. Method for visualisation and analysis of hand and head movements in sign language video. In C. Kirchhof, Z. Malisz, and P. Wagner, editors, Proceedings of the 2nd Gesture and Speech in Interaction conference (GESPIN 2011), Bielefeld, Germany, 2011. Available online as http://coral2.spectrum.uni-bielefeld.de/gespin2011/final/Jantunen.pdf.
Info

2010

Tommi Jantunen, Markus Koskela, Jorma Laaksonen, and Päivi Rainò. Towards automated visualization and analysis of signed language motion: Method and linguistic issues. In Proceedings of 5th International Conference on Speech Prosody, Chicago, Ill. (USA), May 2010. Available online at http://speechprosody2010.illinois.edu/papers/100006.pdf.
Info

2008

Markus Koskela, Jorma Laaksonen, Tommi Jantunen, Ritva Takkinen, Päivi Rainò, and Antti Raike. Content-based video analysis and access for finnish sign language – a multidisciplinary research project. In Proceedings of 3rd Workshop on the Representation and Processing of Sign Languages at 6th Language Resources and Evaluation Conference (LREC 2008), pages 101–104, Marrakech, Morocco, May-June 2008. Available online at http://www.lrec-conf.org/proceedings/lrec2008/workshops/W25_Proceedings.pdf.
Info