How Machine Vision Is Changing Sports Spectating

Technology targeted towards improving audience experience in sporting events is becoming more and more common. Let's take a look at a few examples.

As electronic sensors and sensor data algorithms continue to progress, we’ve seen drastic advances in fields like automated cars, smart devices, and even security through facial recognition and fingerprint analysis. One surprising outcome of this growth is that athleticism has benefitted immensely.
Athletes in training now have access to more discrete biosensors, such as heart rate monitors or sweat analyzers, that can fit on the back of a watch. Just this summer, we ran an article shining a spotlight on technological advances present at the Olympics. It’s all well and good that this technology is benefiting athletes, but what about spectators? Could advances in technology bring about changes in the way we view sports?

Consider how this already occurs in Formula One racing. In the sport of racing, the “athletes” are the cars. By default, the cars are covered in sensors to detect things like brake temperature, wheel speed, and engine performance so that teams can have accurate information about how they are performing. In recent years, that data has been made more and more available to the public, with news channels sometimes showing informative overlays that detail how the car is running. In practice, data is easier to gather on a car, but what’s stopping us from doing the same with human athletes?

Advanced Replays

As it turns out, there isn't much standing between us and truly holistic data gathering for sports. Several tech companies have attempted to change the way we view sports in the past few years, among them Intel and Hawk-Eye. Each of their innovations serves a distinct purpose, but each of them adds a significant new element to their respective sport.

Video content courtesy of Intel.

If you were keeping up with the 2016 MLB season, you may have noticed that the name “Intel” came up a lot, especially during replays. That’s because, this year, they played a large part in creating a new viewing experience: 360-degree replay.
This method of video capture (Dubbed “FreeD” by Intel) is often referred to as “bullet time”, made popular by the Matrix movies. However, even if the theory is the same, Intel’s system is far more complex in practice. The video is captured by 28 precisely-calibrated 5K cameras around the stadium and the image is merged into a three-dimensional view through image-processing algorithms.
This technology has been around for a few years—it started in basketball but has already expanded into both baseball and tennis. The system clearly isn’t perfect, but looking at a FreeD video from even two years ago, viewers can easily see the drastic improvements Intel has made to their system.

Inertial Measurement

Along with popular team sports, Intel has been targeting extreme sports. Intel debuted their onboard data collection system at the Winter X Games this past winter and showed it off again in the summer counterpart. The system consists of a battery-powered system based on their Curie module.
The Curie module is a strange microcontroller that has onboard hardware specifically for capturing inertial measurement via an accelerometer and gyroscope. It’s crazy to think that the same technology that Intel is using to create accurate orientation views of athletes is available on an Arduino board.

Video content courtesy of Intel.

If you’re like me, lots of those tricks looked about the same: someone goes off a ramp and does an impressive series of fast rotations. So, if you’re like me, Intel’s diagrams would be immensely helpful in actually understanding how these maneuvers are quantified and judged. Of course, these algorithms aren’t perfect, either, so judges ignore them when assigning scores.

Ball Tracking

Not all of the technological improvements are all about the viewers; some are about following rules. Companies like Hawk-Eye and Sportvision have created image processing software that can track a ball and show its trajectory through space.

A tennis bounce replay. Image courtesy of Hawk-Eye Innovations.

These replays can improve audience experience, but using the replays as the end-all of judging raises several problems. Several academic studies have thrown doubt on the results of these programs.
The crux of the problem lies in statistics: computer vision is inherently statistical, so any computer vision determination will have error. The software companies present their results as reality and, for most viewers, close is close enough. But when millions of dollars are on the line and the decision is made by a system with potential error, some questions need to be asked. Clearly too much reliance on this tech is ill-advised at this juncture.


With the advent of new technologies such as virtual and augmented reality, the way we spectate is about to be changed. In the next few years, insight into how athletes do the things they do is going to be more available to audiences, and a deeper understanding of any sport is something we could benefit from.