Making Use of a PC Computer to Read Braille

Why would you want a computer to read Braille? I’m working on a Braille ebook reader and I need to ensure its reliability over many refreshes. To start with I only need to verify the position of 12 pins (2 characters of 6 pins), but as the project progresses I’ll need to read multiple lines of 40 characters.

One option is an electrical system, but that would be complicated to build and then need changing as the system grows. I thought I’d start off with a camera based system: take a photo of the pins and then determine if they’re up or down.

Braille and Your PC
Braille and Your PC

Unfortunately, as the pins are quite small (1.5mm diameter) and dense (pin spacing is 2.5mm) the positioning of the camera is critical. And as we’re constantly making changes and taking the machine apart I wanted a system resilient to small changes in the camera’s orientation.

So I decided to investigate using opencv (a free computer vision library) to process the images returned from the camera. I’ve written up my journey here as a reference for myself and others, and because I find writing a good way of checking my own comprehension.

If you’re interested in following along, you can download my scripts and images from github. You’ll almost certainly need to compile opencv from source because the feature detection stuff isn’t in the latest packages. See here for instructions.

I’m a beginner with computer vision, and found the process quite difficult. For one thing, the documentation is fairly arcane and full of definitions I had to keep reading in wikipedia. An added complication is that cv2, (the modern opencv library for Python), uses numpy for all the array handling – which took me a bit of getting used to as well.

As an aside, I chose a second hand Canon power shot as it’s easily compatible with gphoto2, and set it up with fixed focus and large depth of field to keep the pins in focus. From 20cm away, the pin’s shadows are roughly 10px in diameter. I soldered wires to the battery terminals to avoid interrupting long test sequences due to flat batteries. The settings are applied and a photo is taken with this script.

To check if the idea had any chance of working I mocked up some large (3mm) pins. The photo shows 2 empty pins, one pin high and 2 low. The shadow from the pin is easily visible and I thought this would be the perfect indicator for the status of a pin.

A fantastic early discovery was these fantastic tutorials for using the opencv Python library. Most of my programs are just copies of the example code with minor changes. Thanks Alexander Mordvintsev & Abid K!

The chessboard in the picture above is how we can calibrate our particular camera. Opencv assumes that cameras are perfect pinhole cameras, but in reality all cameras have some distortion. Luckily for me, there is a tutorial all about calibrating your camera and then undistorting images taken with it.

Unfortunately the code in the tutorials wasn’t working and it took me a while with ipdb to figure out that a couple of functions had changed their return values. One of the great things about open source software is that it allows people to easily get involved and fix issues. So I was able to fix the document for future users!