3D Controller Doubles as a Functional Computer Mouse

David Liu sat down at his computer, his wrist still aching from whatever had ailed it, and picked up a Kensington SlimBlade Pro trackball in the hopes that it would provide some much-needed relief from all the mouse clicking. Four months later, this became a completely new way to manage 3D applications. Simply roll the ball back and forth like you would on a desk to rotate and move objects with natural ease.

He’d always been fascinated about the two optical sensors that came with this peripheral, which Kensington had begun installing in 2009 to allow users to browse through documents using the twist gesture. But he believed they were capable of much more. They tracked every movement of the trackball in three dimensions; all he needed to do was figure out how to extract the raw numbers. Liu took a close look inside the trackball, scrutinizing the signals originating from the optical sensors. He created new firmware that extracted all of the rotation data directly from both sensors, eliminating the factory chip’s limited output, which only sent out simple cursor motions or scroll steps. He slapped on a bespoke circuit board to run the new software, and before he knew it, he had everything straightened. Plug it in, and a digital cube began spinning as his hand moved the physical ball, which was strange.

Getting it to function perfectly, however, required weeks of careful adjustments, as he’d roll the ball slowly and then flick it hard, altering numbers until the on-screen movement matched the feel of his fingertips. What he actually needed was position control; a ball moved an inch, and the model turned an inch. No more experimenting with a joystick and hoping for the best. Accuracy arrived without continuously correcting oneself, and the next thing he knew, translation was added. To do so, he just held down one button and the ball slid the view forward, backward, or side to side. Pushing the ball’s edge away causes the model to drift deeper into the screen; pulling it towards you causes it to approach closer. After a few tries, he realized that rotation and movement could be controlled with one hand and six degrees of freedom.

As an added bonus, he retained the trackball’s original mouse function intact, which moved the cursor and clicked like a standard mouse. Press and hold control to rotate the 3D model, then shift to pan or zoom the view. You could keep your left hand on the computer for shortcuts and only utilize one ball for the entire task. Choice was good, but now you didn’t even need to select between a mouse and a dedicated controller, because one ball accomplished everything. Software integration improved the whole experience; in Fusion 360, the controller would read the cursor position and simply orbit the model around whichever surface was beneath the pointer; this also worked in Blender, FreeCAD, and Onshape when he wrote little hooks for each software. A model would remain centered where your gaze was directed rather than floating off-screen every time you turned it a small amount.
[Source]