Immersive multimedia and video processing


Today’s advanced multi-media technologies can leverage high-definition audio-visual content, while solutions for virtual and mixed reality allow users to experience new artificial dimensions that coexist with our real world.

The design and implementation of audio-visual computing and immersive multi-media applications is focused on innovative and rapidly evolving technologies, which ask for specific research and development skills. ISIN competences include experience with technologies for the compression, streaming, and manipulation of digital medias such as audio, images and video; solutions for the implementation of 3D immersive multi-media and augmented reality worlds; digital game development with particular emphasis on serious and educational games.

Audio, imaging and computer vision

The availability of high-quality sensors such as video cameras and the proliferation of high-definition audio-visual contents delivered as digital data streams, introduce new requirements in terms of innovative software technologies for audio processing, imaging and computer vision.

ISIN competences in multi-media signal processing range from the development of embedded applications for real-time audio/video processing, including computer-vision systems, to the implementation of stream-processing solutions for large multi-media systems.

Audio processing

  • Development of algorithms for audio synthesis and digital-audio manipulation.
  • Design and implementation of audio-processing applications for embedded devices and workstations.
  • Development of audio-streaming and -processing software infrastructures.


Imaging and computer vision

  • Development of algorithms for digital-image segmentation, object tracking, classification and video processing in general.
  • Design and implementation of real-time video-processing applications for computer vision (including applications for embedded system on chips).
  • Development of video-streaming systems.

Virtual and augmented reality

Virtual Reality (VR) is a branch of computer science dedicated to the creation of synthetically generated environments that provide sensory feedback and give the illusion of being in another real or imaginary place. Artificial stimuli are simulated, for example, through real-time computer-generated graphics (using immersive devices such as large stereoscopic screens, head-mounted displays or CAVE systems), spatial audio, and force-feedback (thanks to haptic devices).

Augmented Reality (AR) and Mixed Reality (MR) are two specific cases of VR where, respectively, new information enriches reality and computer-generated content interacts with the real world.
ISIN experience in VR ranges from the development of its basic components to the deployment of full simulation systems:

  • Design, implementation, and deployment of customized real time 3D visualization software on immersive devices.
  • New human-computer interaction paradigms based on gesture recognition and body tracking.
  • Creation and navigation in interactive virtual environments (e.g., for simulation/training, cultural heritage, entertainment, tele-presence, etc.).

Gaming

The development of modern video games is based on advanced solutions for 3D graphics and for simulating the physical surrounding environment. These functionalities are provided by development frameworks called game engines, which allow for designing and implementing features rich entertainment and serious games. Moreover, modern video games are highly interactive and provide multiplayer functionality over the internet, by using centralized cloud-based servers.
ISIN competences in game development include skills in:

  • Design and implementation of entertainment and serious videogames by using game engines.
  • Development of cloud-based server-side infrastructures for multi-player videogames.

Further information

Tiziano Leidi, Lecturer and Researcher

Achille Peternier, Lecturer

st.wwwsupsi@supsi.ch