Global structure of multimedia system

A multimedia system can be viewed at three levels, called domains: the application domain, the system domain and the device domain. The global structure shows how an application at the top uses system services in the middle, which in turn use the hardware devices at the bottom.

 +-----------------------------------------+
 |  APPLICATION DOMAIN                     |
 |  Applications (e-learning, video conf.) |
 |  User interface | Documents | Toolkits  |
 |  Abstractions for programming (API)     |
 +-----------------------------------------+
 |  SYSTEM DOMAIN                          |
 |  Operating system (scheduling, I/O)     |
 |  Communication system (protocols, QoS)  |
 |  Database / file system                 |
 +-----------------------------------------+
 |  DEVICE DOMAIN                          |
 |  Capture: camera, mic, scanner          |
 |  Storage: HDD, CD/DVD, flash            |
 |  Output: monitor, speaker, printer      |
 |  Network adapters                       |
 +-----------------------------------------+

Application domain

This is the part the user sees. It contains the multimedia applications (video conferencing, video on demand, e-learning, games), the user interface, the document and presentation formats, and the programming abstractions (APIs, toolkits, authoring systems) used to build applications. It decides what the user can do, without knowing how the hardware works.

System domain

It gives the services between application and hardware. It has:

• Operating system: process management and real-time scheduling, resource management, device drivers and file system.
• Communication system: network protocols, transport of continuous media, quality of service (QoS) guarantees and synchronisation.
• Database system: storage, indexing and retrieval of large multimedia objects.

Device domain

It covers the physical devices: input devices (microphone, camera, scanner), output devices (monitor, loudspeaker, printer), storage devices (hard disk, optical disc, flash) and network hardware. It also contains the basic concepts of media representation such as sampling, digitisation and compression hardware.

Flow of operation

A user action in the application (play a video) is turned by the API into a request to the system domain. The OS schedules the CPU and disk, the communication system fetches the stream, and finally the device domain decodes and displays it on the screen and speaker. Each upper domain depends only on the interface of the lower one, so devices or protocols can change without rewriting the application.

Why the layered view is useful

• Separation of concerns: application designers need not know hardware details; hardware makers need not know the applications.
• Portability: the same application runs on different devices if the system domain offers the same interface.
• Resource handling at the correct place: real-time deadlines and QoS are handled in the system domain, because they involve the CPU, disk and network together.

Example: video on demand

• Application domain: the viewer chooses a film from a menu and uses play/pause buttons.
• System domain: the server database finds the file, the network protocol streams it with a guaranteed rate, and the client OS schedules the decoder.
• Device domain: server disks, network cards, the client's graphics card, monitor and speakers.