Using video annotation as a tool for digital humanities awareness

On March 15, 2022, we were invited to participate in the 5th workshop organized by the Digital Humanities in Nordic and Baltic Countries (DHNB) at the University of Uppsala, Sweden, entitled: “Higher Education Programs in Digital Humanities and Social Sciences: Challenges and Perspectives“ Michaël Bourgatte & Laurent Tessier

presentation

During this workshop, we addressed the issue of developing video annotation technologies and showed how the state of the art revealed that there are very few – if any – relevant technologies for exploring film and video in education. We then addressed the issue of developing video annotation technologies for education in the way of the design-based-research protocol (Sanchez, Monod-Ansaldi, 2014). Finally, we focused on the design of the Celluloid project, chosing a case scenario allowing us – by the example – to bring answers to these 3 questions: how to make the analysis of videos enter the field of Digital Humanities? How to encourage the discovery of the cinematographic heritage in an educational environment? And finally, how to accompany learners in developing a critical approach to images in an inductive way?

Why question the status of video in the field of Digital Humanities? Because culture today is also very largely outside of the book. This is what the Swiss researchers Claire Clivaz and Dominique Vinck reminded us in an important article in the French-language journal Les Cahiers du numérique: “Des Humanités délivrées/Humanities Unbound” (2014). In this article they highlighted an important issue for Digital Humanities. Culture is no longer only in books as it was the case before the advent of digital technology. Culture passes through images or sound; culture is multimedia. And one of the fundamental challenges for Digital Humanities is to seize these new forms of literacy “outside the book” – as Claire Clivaz and Dominique Vinck said (2014).

1-the state of the art

The first goal for the Celluloid project was to list – over a long period of time – the existing audiovisual annotation apps and softwares to make an overview of their functionalities and to understand their general philosophy. Several dozen technologies were identified: most of them taking the form of a proof-of-concept in the context of academic projects. They were not being used on a long-term and sustained basis.

In France, the annotation software that has been most intensively used in an educational context is Lignes de Temps, a very similar technology to another service perhaps better known abroad: Anvil. This software was developed at the Centre Pompidou. Designed in collaboration with film professionals, it was inspired by movie editing applications – as its name implied (“Lignes de temps/Timelines”) – focusing on the time-related aspects of films and video. A software that allowed users to take notes, add keywords, put tags, etc. In a nutshell: a set of useful features in pedagogy, including with first-grade students. A software that can however only be used alone and not in the context of distance learning or mix-pedagogy. Moreover, it was a software, which means that it cannot be used online.

Other technologies are making a difference by offering a view of the screen with the player on a side and annotations on the other one, rather than a timeline. The best known webservice is Vialogues developed at the Columbia University. A webservice that has undoubtedly been developed for educational purposes that also has its limits. First, you can only work on Youtube videos and the annotations are not associated with the images (the links is based on a timecode). Secondly, one is forced to connect and therefore to have an email address or a Facebook account, which is not adapted to young audiences.

A sum of elements can explain why video annotation practices had not been developed in educational contexts. Interviews with teachers and researchers confirmed it. They were largely ignoring the existence of these tools. And when they are aware of them, they either distrust them because they are unreliable, or they turn away from them because they are poorly adapted to their teaching practices. Most of the time, they are satisfied with sharing sequences using a simple video player such as VLC (occasionally using bookmarks to explore the audiovisual content).

This state of the art has revealed that the usability and technological choices made by the software engineers and developers slow down the collaborative dynamics of groups; they do not allow to distinguish the status of users (especially between teachers and learners); they are not adapted to distance collaborations either.  This has also allowed us to reveal the great difficulties that teachers and researchers face in setting up educational projects using video.

2- an experiment promoting interaction around the video

From there, we developed a video annotation application that is the outcome of our observations and ideas about (1) design thinking, (2) collaborative work and finally (3) the Digital Humanities culture, regarding the open-source features and the free use of the service by whoever wants it. This allowed us to achieve the three goals we had set for ourselves:

– To be able to annotate directly on the content. For this, we have associated basic functionalities of a player (play, pause, stop) with a toolbox allowing the annotation work to be carried out, eg.: inserting a text, an image or a hyperlink.

– Our second goal was to facilitate interactions. To do so, a teacher or trainer must be able to give instructions, answer questions or initiate discussions, while the learners must be able to annotate the videos, either by giving answers or by asking questions to the teacher. The webservice must therefore support the collaboration of several participants with different status.

– Thirdly – and probably in a more abstract or conceptual way – we wanted to desacralize the film or the video. Based on the practices of textual annotation that became more democratic with the birth of the paperback book, we decided to promote an annotation practice carried out directly on the film. Technically, we insert annotation on a frame that is put on the image. At the end, we obtain a series of annotations that we can make appear or hide as we wish. As surprising as it is, it was Youtube that gave us this idea. Indeed, the platform has offered this functionality for a long time before removing it.

3- birth of the Celluloid annotation service

This video annotator we developed is named Celluloid (available on celluloid.huma-num.fr). A teacher creates an account, then creates a project containing a video and some instructions.

Today, the video is streamed directly from university computers servers (this was made possible by the teams of the La Rochelle University, France which we would like to thank, especially Cécile Chantraine Braillon who is in charge of the development of the app). But we are currently working on the development of a network of linked servers to facilitate the exchange of audiovisual resources between universities and schools. In France, we seriously consider the idea of using the Nakala sharing data service developed by Huma-Num (an institute supported by the French public research organization – CNRS) We can also consider the Frama network of freelance developers and researchers – which has developed a distributed streaming service called Peertube. One of the interests of the Peertube webservice is to offer a search engine which looks similar to the Youtube one – with the difference that the system is not centralized but distributed.

Once the teacher has created his or her project, he or she shares it with the learners. A scenario we have imagined is the one of a Paula, a history teacher who wants to work on the 1919 Robert Wiene’s movie The Cabinet of Dr. Caligari. In her course, Paula wants to talk to her students about the work of Siegfried Kracauer, a german sociologist who saw the rise of Nazism in the German Weimar Republic expressionist cinema (1947). In order to do this, Paula wants to use an inductive method by getting her students to hypothesize and experience this theory firsthand. Thus, each time a new character appears on the screen, Paula makes an annotation in the form of a hint, for example: “describe the expression of this character”.

Once this work is done, Paula shares her project with her students by giving them a password. The goal is to speed up the work process by avoiding the need for students to identify themselves individually and to face common problems today: loss of login data or fear of leaving web traces (which is particularly concerning when working with children). All they have to do is log in, then get to work answering questions and collaborating with their classmates. A few days later, Paula logs on to the platform and reads the annotations that have been produced.  She answers some of the questions with new annotations and she selects a number of them to organize a discussion during her next course. Back in the classroom, and following the rules of the reverse-pedagogy, Paula broadcasts movie clips with a VP and provokes a discussion based on the annotations.

references

Clivaz, C. & Vinck, D. (2014). Introduction: Des humanités délivrées pour une littératie plurielle. Les Cahiers du numérique, 10, 9-16. https://doi-org.icp.idm.oclc.org/10.3166/LCN.10.3.9-16

Kracauer, Siegfried. (1947). From Caligari to Hitler. A Psychological hHstory of the German Film. D. Dobson.

Sanchez, Éric et Réjane Monod-Ansaldi. 2015. « Recherche collaborative orientée par la conception. Un paradigme méthodologique pour prendre en compte la complexité des situations d’enseignement-apprentissage ». Éducation et didactique 9 (2) : 73-94. https://doi.org/10.4000/educationdidactique.2288.

Tessier, L., & Bourgatte, M. (2017). Les outils d’annotation vidéo pour la recherche. In Cavalié, É., Clavert, F.,

Legendre, O., & Martin, D. (Eds.), Expérimenter les humanités numériques : Des outils individuels aux projets collectifs. Presses de l’Université de Montréal. doi :10.4000/books.pum.11100 – https://books.openedition.org/pum/11100?lang=fr