On 23 February 2016, FEP participated in the final conference of the EU-funded project CRe-AM (Creativity Research Adaptive Roadmap), concluded in September 2015 and aimed at engaging the creative industries and creators with ICT researchers and developers and drafting a series of roadmaps for developing new technologies for the creative sectors, from which to define recommendations for policy, planning, and decision making for the creative industries, focusing on the future ICT R&D agenda. The event opened with a presentation of the roadmapping methodology, based on the elaboration of data from extensive interviews to expert sectors to track emerging paradigms related to mindsets, techniques and technologies and thus map trends, strengths and weaknesses and identify areas for further development. The following sessions provided details about how the methodology had been applied to several creative sectors (including media and e-publishing) to highlight challenges and propose specific research directions; the exercise, in many respects similar to that of the TISP project, identified many interesting visions and technological trends for the publishing sector, such as IPR protection, content personalisation and big data analysis, and metadata.
The CRe-AM final event was called ‘ICT and creative industries in Europe: current trends and future policies’. The project was meant in fact to connect communities of creators with communities of technology providers and innovators, in a collective roadmapping effort, with a view to identify and stimulate the development of ICT and tools addressing the needs of different sectors of the creative industries.
In the introductory session, Carl Smith and Jazz Rasool (Ravensbourne, UK) presented the project, stressing its aim to design strategic roadmaps for developing new technologies for the creative industries and explaining that it had looked at all creative industries and technologies, to carry out mapping exercises that could be useful for each sector and across sectors.
The roadmapping strategy was based on a methodology that, according to its developers, would enable experts to intuitively see state of the sector, by identifying current centres of gravity for the industries and hidden interferences that revealed potential changes in mindsets, techniques and technologies; the methodology, being numerically underpinned, could be used for forecasts, it allows exploring scenarios and simulations and is repeatable and possibly commercially valuable.
The process relied on a theoretical model to track emerging paradigms in science and society, the Emerging Paradigm Model, which looks at cycles of three integrated phases: mindsets, techniques and technologies, to identify weak and strong signals and trends in specific sectors. The project looked at architecture, media & e-publishing, art, gaming and design. The methodology consisted first in gathering several expert opinions, via interviews on the current situation and expectations about the future; techniques and technologies were identified and the opinions expressed were turned into numbers, to then apply the strategy in its three components: the Emerging Paradigms Model, the technology trends and the signals matrix.
The Emerging Paradigms Model was used to look into mindsets (as a change in mindset often favours innovation), techniques or practices and technologies; these led to identifying technology trends, analysing them and detecting their implications and related opportunities, to find out how to leverage them and how to mitigate the associated risks; this in turn allowed to draw a ‘signals matrix’ for each sector, a colour-coded hidden map that provides a visual description of the sector and allows comparisons across sectors. Created by turning the expert opinions into numbers and the numbers into sector overviews, the signals matrix highlights environmental factors, pressures, structural forces, supportive forces and the resulting net forces (pressure/support, strong/weak points). This is the basis for drawing a signals matrix roadmap, where techniques and technologies are arranged hierarchically along the vertical and horizontal axes; strong elements need to be present at the bottom of the matrix to signal a sustainable situation. By adding up strengths and weaknesses from all neighbouring areas for each element of the matrix, a ‘ pressure matrix’ is obtained, that shows overall strengths and weaknesses in a sector; by comparing this to the original matrix, a ‘trends’ matrix can be drawn, which allows in turn to identify centres of emerging trends. Finally, the trends can be used to build a ‘three horizons chart’ that explores 3 time horizons (current arena, near future, far future) and can be used to make forecasts, identify societal trends and provide recommendations for funding allocation.
Basically, the methodology led to drawing maps for each sector that identified strong signals (technology trends) and weak signals (the very first signs that something will become significant and should be supported in its development) in the context of new/emerging technologies in the creative sectors, highlighting technology gaps between desired future technologies and probable future technologies and building roadmaps providing specific recommendations for stakeholders and policy makers about the technology areas that should require support in the near future.
The following sessions looked into the results of the work on each of the sectors involved. Fabrizio Giorgini (R&D Manager at eXact Learning Solutions) presented the roadmap for the media and e-publishing sector, consisting of three components: visions (and desired future scenarios), trends and weak signals, and challenges and research recommendations.
The visions and desired future scenarios were derived from data collected at events and from individual interviews with experts, who were asked to define future scenarios (what they expected for the next 5-10 years). The most popular visions were selected and the related technology trends and weak signals (emerging areas in need of development to support related trends, which could undermine further developments if not followed adequately) were studied; these are the results:
- Vision 1: Creative Production. Experts envisaged the introduction of content formats and technologies aimed at streamlining many processes from content creation, editorial and post-production through to broadcast and distribution, leaving humans to focus on the creative and value-added aspects and collaborative production and development.
- Related trends: automated workflow and metadata technologies (automated content annotation and data management based on Artificial Intelligence, automated content discovery); contextual citation and annotation (semantic ontologies, standardised annotation technologies, social reading& shared bookmarking); Open Access and standards (open data and formats, common file formats such as ePub 3 and 4, open indexing standards).
- Related weak signals: file format developments (beyond PDF; portability of content across devices and platforms and interoperability between platforms, devices and software formats).
- Vision 2: Interactions & Engagement. Experts indicated an interest in technology aiming to provide new forms of real time audience interaction, and that enable more dynamic engagement of the end-user.
- Related trends: immersive user interfaces (Augmented Reality & Virtual Reality, audience interaction technology, sensors and data-driven interactions); interface technology (embedded sensors in physical environments).
- Related weak signals: tactile internet (integrating real world objects with computer generated content); better display technologies (personal projectors in mobile devices, holographic interfaces); interlinking of content and data (allowing readers to interact with data, not just to read about them, and chunking of data to provide modular content).
- Vision 3: Personalisation. Experts greatly emphasised personalisation and customisation of content based on users’ profile, emotions and behaviour, to produce a more adapted user experience, which could be embedded within existing devices and content platforms, as well as Augmented Reality and Virtual Reality technologies.
- Related trends: big data analytics (data mining of user profiles and behaviour, real time data visualisation).
- Related weak signals: mood and emotion tracking.
- Vision 4: Infrastructure and Archiving. Experts indicated the need to more seamlessly broadcast and distribute contents as well as the ability to efficiently and reliably archive and preserve digital contents.
- Related trends: 5G communications network (reduced latencies, very high bandwidth); Cloud; data compression (real-time).
- Related weak signals: generalised video codec (for compressing/decompressing digital video).
- Vision 5: Digital Rights and IPR Protection. Experts highlighted the need to improve the IPR management, protection and exploitation with various mechanisms (e.g. new protection and tracking systems and standards). This was a key point for both the media and e-publishing sectors, and was seen as issue placing many constraints on innovation
- Related trends: IP, rights and management (shared standards for file migration).
- Related weak signals: end to end IP (real time tracking of content rights, tracing unlicensed content usage); Blockchain technologies (standardising IPR/DRM deployment).
The visions, trends and weak signals were the basis to identify challenges and make specific research recommendations (including the expected timescale of developments), as follows:
- Challenge 1: Content production. Research directions:
- Improve collaborative production tools: tools that enable creative professionals to combine live performances, video and computer-generated imagery in real time and collaborative to create imaginative entertainment and experiences; timescale: within 5 years.
- Automated content integration: data sources and services specifically designed to support machine-to-machine access to content, including a comprehensive suite of APIs and content syndication services (APIs should be using standardised formats for indexing and retrieving contents); timescale: present.
- Automatic translation (real time translation tools that overcome linguistic, cultural and disciplinary barriers); timescale within 5 years.
- Challenge 2: Content annotation and discovery. Research directions:
- Artificial Intelligence for automatic annotation; timescale: beyond 5 years.
- Metadata standards for content description: there are several metadata standards (Dublin Core, METS, MARC, ONIX, etc.) to describe, locate, purchase and recommend books and e-books, hence the need to define one unified metadata standard that also allows the description and consequent reuse of content chunks (chapters, paragraphs, images, etc.); timescale: unclear.
- Challenge 3: Immersive and interactive user interfaces. Research directions:
- Improved Virtual Reality: improved performance in the synchronisation between the user’s head movements and a visualised picture, also through a symbiotic relationship between content and technology to overcome problems like nausea induced by the use of VR even for short periods, and improved sense of presence by adding smell and touch (currently at prototypal level); timescale beyond 2 years.
- Gesture recognition: the ability to recognise detailed gestures to provide a vast new range of screenless interactions; timescale: present.
- Interactive videos: technology that makes watching a video as immersive as navigating a website, something the user can interact with, “played anywhere” and able to serve user-tailored content thanks to data analytics on user behaviour; timescale: present.
- Challenge 4: Content personalisation. Research directions:
- Users’ behaviour modelling: big data analytics and data mining for modelling users’ behaviour, intelligent transformation and interpretation of large amounts of behavioural and asset-based data, enabling a better understanding of the audience and emerging trends (in real time, whereas now it’s done mostly offline), as well as cognitive analytics to model the way the human brain processes information, draws conclusions and learns from actions taken to generate hypotheses, make conclusions and express recommendations; timescale: present.
- Mood and motion tracking: machine learning and artificial intelligence to identify facial expressions beyond the 8 core emotional states currently recognised, that may also be culturally dependent; timescale: beyond 3 years.
- Challenge 5: Content broadcasting (for the media sector). Research directions:
- Real time video coding: to move toward a unified standard for encoding videos, including 3D multi-view videos; timescale: beyond 3 years.
- Adaptive bit rate: new techniques to improve existing player performance by providing quality-related information to the player, enabling it to make better bit rate choices, resulting in a reduction in overall bit rate requirements and a significant reduction in content quality variance; timescale: beyond 3 years.
- Challenge 6: Content protection (seen as very relevant by both media and e-publishing). Research directions:
- Secure data encryption: adoption of Blockchain or alternative technology to provide new ways to secure digital contents and to track and verify ownership through tools like smart contracts, authenticated by cryptographic data; timescale: beyond 3 years.
- DRM: rights management and royalty payment systems that allow taking into account a business more reliant on chunks or modules than on book products; timescale: beyond 3 years.
The presentation was followed by a debate that tried to draw some conclusions based on several discussion points, focusing on whether the scenarios presented reflected those envisaged and desired, whether the identified challenges correctly represented the main concerns in the sector and what the main recommendations for future research should be. Roger McKinley, Research and Innovation Manager at FACT (Foundation for Art and Creative Technology), was invited as expert discussant. He confirmed that the needs of the sector had been largely well represented, and added a recommendation to look into the civic engagement of consumers and their role as producers and editors. He also pointed out the importance of privacy concerns, especially with regard to content personalisation based on behavioural data.
Among the other sector roadmaps, the one on arts presented some overlapping with media and e-publishing. One of the visions, on archiving and digital preservation, proposed research in smart metadata (for better use of digital media in the creative industries) and automatic file migration; another one, regarding IP and copyright protection, envisaged desired future technologies for embedding copyright and IPR in digital media and developing smart IP metadata to report on IPR violations (again, smart contract technologies), and was the basis for a challenge focusing on IP, security and data protection, which proposed research directions such as watermarking technologies.
All in all, it would have been more useful to keep the media and e-publishing (books) separated, since a number of topics were not equally relevant for both sectors (content production and broadcasting), or had different elements of interest (immersive and interactive interfaces, content personalisation and annotation). However, several interesting issues were raised with regard to the book sector, such as content discovery and the related metadata for content description, big data analytics for content personalisation (and not only), technologies for improved rights management. These broadly correspond to topics identified in the framework of the TISP project and included in the project’s policy recommendations, which show a great potential for synergies with the results of CRe-AM.