Libraries, system software, languages, OO

These four are the main software-oriented abstraction levels for programming multimedia systems.

1. Libraries

A library is a set of ready-made functions for media tasks (capture, play, edit, display). The application calls them through an API, without handling the hardware directly.

• Provide device-independent access and save development time.
• Examples: DirectX, OpenGL, SDL, OpenAL, FFmpeg.
• Limits: restricted to the functions provided; usually not integrated with the language, and the programmer must handle timing and synchronisation.

2. System software

This is the operating system and its multimedia services.

• Device drivers for audio, video and graphics hardware.
• Real-time scheduling and resource management to guarantee continuous media delivery.
• File systems and buffer management for large streams.
• Media control interfaces and synchronisation support.
• Examples: Windows multimedia services, Linux ALSA and V4L, Android media framework. Applications use it through system calls or an API.

3. Higher programming languages

Languages with built-in or class-library support for media, hiding system details. Media is handled by simple statements, and synchronisation and timing are supported by the language.

• Examples: Java (Java Media Framework, Java Sound), ActionScript/Flash, Lingo, Python (pygame), JavaScript with HTML5 audio and video.
• Easier and more portable, but less control over performance.

4. Object-oriented approaches

Media, devices and streams are modelled as objects with attributes and methods.

• Encapsulation, inheritance and polymorphism give reuse, modularity and easy extension. For example, a class Media can have subclasses Audio, Video and Image, each with play() and stop().
• Common media classes, device classes and stream classes simplify design; they allow a standard model for creating, processing, and synchronising media.
• Examples: class frameworks and Java classes, C++ toolkits such as Qt Multimedia.