Case Study – Long Duration QA/QC Testing
Quality Assurance (QA) and Quality Control (QC) refer to systematically monitoring and evaluating various aspects of a project, service or facility to ensure that standards of quality are being met. It is important to note that quality is determined by the intended users and thus is not quantitatively definable.
How does a broadcast manufacturer ensure that their processing units have high quality?
To answer this question, we need to define what we mean by processing unit. For the purpose of this paper, we will define processing units similarly to those used in telecommunication:
- a compression device that also acts as a transmitter (video encoder)
- a transmission medium (cable, IP, satellite, microwave, and wireless)
- a decompression device that also acts as the receiver (video decoder)
Also, as is customary, video refers to image & sound or audio & video.
Video processing is a special case of general signal-processing where the input and output signal is a video stream or file. The most common form of video processing is to compress the video to fit it into the available bandwidth. The device that does this is termed a video encoder.
A video encoder reduces the quantity of data by removing the redundant information within a frame and between two successive frames. The digital video formats defined by VCEQ and MPEG are the de-facto standard for broadcast video. They are popular because
- There are no restrictions on the implementation of the video encoder (compression device).
- The video decoder’s (Set-top box, PC) capabilities are fully-defined based on levels and profiles.
- The standards include video, audio, transport, and timing functions.
These video formats include – MPEG-1, MPEG-2 (DVD), H.263 (video surveillance), MPEG-4/H.264 (combined next generation standard), JPEG (still pictures), JPEG-2000 (archival) – just to name a few. With the possible exception of JPEG-2000, all of them are lossy (information is lost during compression so the quality after encoding/decoding is not as good as the original). JPEG-2000 can be lossy or mathematically loss-less.
In practice all lossy video encoders generate artifacts (areas of unfaithful visual/audible reproduction). If the encoder is designed and configured well and the data rate is high enough, then these artifacts will be virtually invisible. But video encoders do not work in a vacuum. Other factors such as transmission systems, temperature, and bad inputs can cause errors even to the best designed video encoder.
Video is transferred over a point-to-point, point-to-multipoint transmission medium. Examples include copper wires, optical fibers, wireless, or storage (DVDs, flash drives, and hard disks).
Even in guaranteed service networks, bit errors do occur. The streams are sent over many routers and any one of them can delay the packets (causing jitter), reroute the packets (causing loss or reordering), or simply fail.
One of the easiest ways to create an error is to oversubscribe (overbook) the network. This is the process of connecting more video streams to the network than can be supported if all of them operate at peak usage. In normal operations, the network never runs at peak usage so this condition is not fulfilled. If it is, then packets are dropped, which will cause the video to stop or skip.
Video decoders fall into 2 categories:
- Professional grade integrated receiver/decoder (IRD). These normally have professional I/O connections like HD-SDI, SDI-3G, DVB-ASI, etc.
- Consumer-grade converter/cable boxes called set-top boxes (STB). They have consumer I/O connections like component, SCART, HDMI, etc.
Video decoders are computerized devices, which receive compressed digital signals, decrypt/decode them, and convert them to either an analog or digital format to be shown on a TV. The video decoder can be either an external box, built into the TV, a PC, a gaming console, etc. Regardless, it makes it possible to receive and display TV signals, connect to networks, play games, and surf the Internet. One of its primary functions is to detect errors, and fix or conceal them. It does this by:
- Holding previous frame/partial picture
- Asking for a retransmission (Microsoft’s IPTV solution)
If the video decoder is processing on screen graphics, it could run out of processing power to decoder/display the video. It could also receive bad video input and shut off.
Pictorial Work Flow
The above picture illustrates a simplified video processing WorkFlow.
An uncompressed video is sent to a video encoder, which compresses the data and prepares it to be sent over a transport media. The video decoder receives the data decodes and decrypts it for viewing on either a monitor or TV.
How do you test in this environment? A natural thought is to send many video streams into the processing WorkFlow and to capture, analyze, and view the results.
During this video process, many things can go wrong
- The video encoder can badly compress the video
- The video encoder can package the data incorrectly causing a poor transmission
- The transmission media can add delay or miss a packet
- The video decoder could fail to read the transmitted stream
- The video decoder can badly decompress the video
For shorter duration testing, it is best to use a test & measurement video analyzer that can playout uncompressed video streams while simultaneously recording them. After the stream is recorded, the test & measurement analyzer can process the streams and generate a pass/fail criterion. Video Clarity designed the ClearView Video Quality Analysis Systems for this purpose (www.videoclarity.com/products.html).
What would happen if the video processing units did not produce an error for several hours or days? Perhaps a particular set of input data sent at just the wrong time was needed to create the problem. This type of problem is very difficult to replicate, but it will be the first problem that your customer’s find.
Several video equipment manufacturers have turned to Video Clarity to solve this problem. To test, you need to continuously monitor the video quality, detect the error event, and save the data before and after the event so that you can diagnose the problem and devise a fix!
Video Clarity RTM Solution
Errors will occur. Systems are not perfect. Video Clarity developed a real-time monitor (RTM) that compares 2 video streams. It aligns the audio and video and generates alerts when:
- Video Quality drops
- Audio Quality drops
- VANC data is not complete
- A/V delay is not correct (lip-sync problem)
In addition, it reports
- The average A/V quality
- A/V delay/offset
- Any dropped frames and then dynamically realigns
Figure 1: Hardware Video Quality Monitoring
RTM takes 2 SDI feeds, professional adapters are available for component, HDMI, etc., and it aligns the video and the audio.
The patent-pending alignment looks for areas with temporal disturbance.
- scene changes,
- fades to black, or
- fast motion changes.
- talking to silence,
- talking to music, or
- music to silence.
Regardless, it is not the magnitude of the change as much as the difference from average. For example, a hockey game has a great deal of motion, but the motion is fairly similar. For video conferencing, a camera angle shift could be a huge motion relatively speaking.
Figure 2: Temporal Event in the Video and Audio
The reference and processed audio and video are analyzed separately, and an offset for each is calculated. This offset is the delay caused by processing. In theory the audio and video delay should be equal. If they are not, then the difference could create a lip-sync problem.
If the audio comes out early, then this will create a bigger disturbance as we are conditioned to wait longer for the sound. From basic physics, light travels faster than sound so we see the image first and then wait for the audio. If this ordering is reversed, it bothers us.
RTM can run via SNMP, socket commands, or from its interface (GUI) shown below. All of the data is logged and alerts are generated.
Using RTM, a manufacturer can setup a test and leave. Alerts and logs will be generated if an error occurs based on user defined profiles. The errors are saved (logged) along with the audio and video sequences before and after the alert.
Figure 3: RTM GUI
RTM and ClearView Availability
RTM are ClearView are currently being used by many broadcast equipment manufacturers. To get a demonstration, please contact Video Clarity or one of its channel partners (www.videoclarity.com/sales.html) or visit us at one of the shows listed at www.videoclarity.com.