To be useful, robots need to perceive and interact with the world. Depth cameras are a type of sensor that allows robots to see a scene in a way that fundamentally links it to the geometry of the world.
A traditional camera, or RGB camera, takes a picture of red (R), green (G), and blue (B) pixels that can be imagined like a spreadsheet with 3 values per cell.
An image from a depth camera can be imagined as a spreadsheet. However, unlike with an RGB camera sensor, each cell from the depth image has only one value. This value represents the depth, often in meters, from a camera’s sensor to the object in the 3D World along the axis pointing out of the camera. The underlying technology that makes this possible varies, but one common one is based on stereo vision. If you cover one of your eyes with your hand, do you notice your depth perception suffer?
In this depth image, each pixel has been colorized according to how far it is from the camera. Far pixels are cooler colors like blues, close pixels are warmer colors like reds.
The stereo matching process can make mistakes and introduce noise into the image. In the example image, note that the edges of boxes have shadows and there are splotchy patches in the image. However, unlike with an RGB image where color is the only data, a depth image tells you a lot about the geometry of the world. For example, exactly how close is that screwdriver to the camera?
For this reason, it can be an appealing tool to speed up machine learning. Some tasks that may take large models and a lot of training time with RGB images may be simpler and faster to train using depth images.
For more information on robotics sensors and how Practical Robotics can apply them to problems at your business, please contact us through the form.