If you launched the venerable Windows Media Player in 95, 98 or XP, get ready for gentle shock: : it was lying to you.
And by lie, well, what I really mean is rendering the video somewhere other than in the actual window that was open on the desktop – a sort of parallel plane of existence to the desktop you were actually looking at – before sneakily moving it.
If you’ve watched a single Twitch stream or Marvel movie in the last 15 years, you’re probably familiar with the concept of a green screen, a radiant, single-color surface that makes it straightforward to erase the background and fill it with something else. A Hawaiian beach, a bunch of CGI superheroes punching each other, etc. A more true, color-agnostic term for this technique is chroma key, which is how Windows rendered all its video back in the day.
“The media player program was not rendering video pixels on the screen,” Raymond Chen, a longtime Microsoft developer, recently wrote in a blog post. Instead, Windows would render a green screen (or another color, depending on the version) and then “render the video pixels to a graphics surface shared with the graphics card.” The final step was to “tell the graphics card that whenever it sees a green pixel that needs to be written to the screen, it should replace the pixel from the shared graphics surface.”
Graphics surfaces were typically called overlays because they (invisibly) overlayed the desktop (which would come into effect soon). Chen said the technique has several advantages: Avoidance pixel format conversions if the video format is different from the user’s monitor format, and maintaining consistent performance by separating the graphics surface from the process thread that manages the user interface. So even if the Windows shell freezes for a second, the video will play smoothly.
A more advanced technique, called flipping, would enhance this performance advantage by having two shared graphics surfaces, one with the “current” video frame and the other with the “next” video frame, between which the graphics card would switch during a very vertical blank space (as the screen refreshes).
This venerable video playback technique isn’t very compelling until you figure out how you can intentionally or unintentionally confuse the way Windows uses chroma key, as Chen explained:
“When you take a screenshot, you get the pixels that Windows gave to the graphics card as desktop content. If the overlay is active, these are not the same pixels that came out of the graphics card and sent to the monitor. The computer never sees these monitor pixels; they are generated on the fly by the graphics card and sent directly to the monitor. Your screenshot was a screenshot of a desktop screen and contains green pixels where the video was supposed to go.
“Now, when you load an image into Paint or another image viewer, Windows sends these green pixels to the graphics card, but if the media player is still running, its overlay is still dynamic, and if you place Paint in the same place as the media player window, the green pixels in Paint will be replaced with dynamic video pixels. The graphics card doesn’t know whether the pixels are coming from Paint. Its job is to look for green pixels in a specific area of the screen and replace them with pixels from the shared surface.
“If you move the Paint window to a different location where it doesn’t overlap the media player, or if the media player isn’t playing video, you’ll see the true nature of the bitmap: it’s just a bunch of green pixels.”
Video rendering used to be quite a demanding task for computers, but nowadays there is no need for the kind of techniques that allowed Windows to play a tiny 240p mpeg file at 60 frames per second. I know it’s for the better, but personally I’m a little unhappy that Microsoft Paint is now intelligent enough to remove backgrounds and even save layers like Photoshop. Computers are just more fun when you can trick them.
