Advanced Humanoid Robotic Interface & Control

Visual Perception System: Key to enabling humanoid robotics to perform dexterous tasks, R2’s visual perception system (U.S. Patent No. 8,244,402) – comprised of machine vision cameras, processors, and novel algorithms – allows robots to find, track, and measure objects automatically in their field of view. Tendon Tension Sensor: Unique tendon tensions sensors (U.S. Patent Nos. 8,371,177 & 8,056,423) are embedded in R2’s palms to enable granular force control of the fingers via a redundant network of tendons. R2’s tendons are coupled to, and used to actuate, the robot’s finger joints. Thus, tendon tension measurements provided by the sensor allow for the external loads experienced by its robotic fingers to be derived. Tactile System: R2’s hands feature an innovative tactile system that grant the robot a sense of touch (e.g., measurement of external contact forces, shear force, and slippage of objects held in the hand) – an important requirement for robots designed to perform complex tasks in an automated fashion. The tactile system is enabled by novel six degree of freedom (DoF) force torque sensors (U.S. Patent No. 7,784,363), which are integrated into the fingers at each phalange (14 per hand). A calibration system (U.S. Patent No. 8,265,792) ensures the sensors maintain high accuracy throughout operation. Contact State Estimation: A contact state estimation sensor (U.S. Patent No. 8,280,837), based on the use of a particle filter, enables R2 to perceive the location, orientation, and shape of objects when in contact with the robot’s hands (i.e., tracks hand-object state). The contact state estimation system leverages a novel motion model, which characterizes the motion of a robotic hand as it moves relative to an object of interest. Series Elastic Actuator (SEA) Sensing: R2’s SEAs achieve fine torque sensing at each of its joints without sacrificing strength or payload capacity. The robot uses two 19-bit absolute angular position sensors, calibrated using a novel technique (U.S. Patent No. 8,250,901), to measure the deflection of each spring in real time.

The R2 hand and forearm assembly represents the cutting edge of humanoid robotics technologies. The highly modular design provides significant improvements over prior humanoid robotic hands, especially in the areas of strength, speed, sensing, and ability to approximate human grasps. Hand, Finger, and Wrist Assembly Design: The robotic humanoid lower arm design (U.S. Patent No. 9,505,134) includes novel robotic finger (U.S. Patent No. 8,562,049), thumb (U.S. Patent No. 8,424,941), and wrist (U.S. Patent No. 8,498,741) assemblies. Actuation & Control System: A novel finger actuation system (U.S. Patent No. 8,467,903) comprised of an actuator, tendon, conduit, tension sensor (U.S. Patent No. 8,371,177), and terminator is perhaps the primary enabling technology for R2s compact, high performance robotic hand. The actuation system is packaged in the wrist (U.S. Patent No. 8,401,700) and reduces the number of actuators, providing significant space savings. Control systems include methods for tensioning (U.S. Patent Nos. 8,412,376, 8,618,762, & 8,056,423) and controlling torque (U.S. Patent No. 8,565,918) of the tendon-driven robotic fingers. The finger actuation control system (U.S. Patent No. 8,489,239) can operate using force- or position-based control laws. Tactile System: R2s hands feature an innovative tactile system that grant the robot a sense of touch (e.g., measurement of external contact forces, shear force, and slippage of objects held in the hand) an important requirement for robots designed to perform complex tasks in an automated fashion. The tactile system is enabled by novel six degree of freedom (DoF) force torque sensors (U.S. Patent No. 7,784,363), three of which are integrated into the fingers (at the proximal, medial, and distal phalanges) and two in the thumb (medial and distal phalanges). A calibration system (U.S. Patent No. 8,265,792) ensures the sensors maintain high accuracy throughout operation. Autonomous Grasping: A novel grasp assist device (U.S. Patent No. 9,878,452 & 9,067,319) enables reliable, autonomous interaction with a broad range of objects (e.g., tools). A particle filter-based contact state estimation device (U.S. Patent No. 8,280,837) performs object localization and characterization.

R2's unique systems allow the robot to be used in many telemedicine applications and in many medical scenarios. For example, R2 can assist a surgeon and the surgical team before, during, and after a procedure with multiple tasks. The robot has the vision, dexterity, and the ability to perform tasks tirelessly 24 hours a day, seven days a week. R2 can work safely around humans, so it can be integrated into a dynamic hospital environment. The R2 technology capabilities in telemedicine are being explored through partnerships with medical institutions. After a quick medical procedure training, a R2 teleoperator was able to guide the robot and perform an ultrasound scan on a medical mannequin. Humans at the controls were able to guide the robot to perform the task correctly and efficiently by using R2's dexterity to apply the appropriate level of force and were able to track their progress using the robot's vision system. The technology was also used to experiment using a syringe and an intubation procedure with a mannequin to demonstrate R2's telemedicine capabilities. R2 is well suited to be used by physicians to conduct medical procedures on humans in remote locations.

R2 uses brushless DC motors, harmonic drive gear reductions, and electromagnetic failsafe brakes as the building blocks for the powerful, torque-dense actuators in its human-scale, 5 DoF upper arms. Moreover, the use of series elastic actuators and novel tension sensing & control systems represent some of the most innovative technologies present in the humanoid robotic arms of R2. Series Elastic Actuators (SEAs): R2’s SEAs achieve fine torque sensing at each of its joints without sacrificing strength or payload capacity. Such capabilities are enabled through the development of several advanced technologies. Specifically, novel planar torsion springs (U.S. Patent No. 8,176,809) are integrated into each rotary series elastic actuator (U.S. Patent No. 8,291,788), while two absolute angular position sensors, calibrated using a novel technique (U.S. Patent No 8,250,901), measure the deflection of each spring. Force and Impedance Control Systems (U.S. Patent No. 8,525,460): These systems use position sensor signals for sending position data to an embedded processor that determines the positional orientation of the load relative to a motor shaft and its related torque on a string. A FPGA-based controller (U.S. Patent No. 8,442,684) provides a high-speed (10 KHz) control loop for the electric motor and gear reduction assembly present in R2 joints. Tension Sensing & Control of Tendon-Based Robotic Manipulators: NASA has also developed technologies to provide tension sensing & control of humanoid robotic arms. First, a tendon tension sensor (U.S. Patent No. 8,371,177) measures strain on tendons (strings) employed in robotic arms. A novel calibration system (U.S. Patent No. 8,412,378) calibrates the tendon tension sensors. Finally, joint space impedance control systems (U.S. Patent Nos. 8,170,718 & 8,060,250) provide closed-loop control of joint torques or joint impedances without inducing dynamic coupling between joints, as well as programmable Cartesian arm stiffness.

While robotic technologies are already being used in several industries like logistics and distribution, R2 allows for much more complex and delicate operations that require a more sophisticated level of interaction. In terms of handling inventory, R2's dexterity would allow it to handle a multitude of items, including delicate ones. In addition, it can perform in close proximity to humans, allowing for the use of robotics in areas where it's not currently safe or practical. R2 is equipped to navigate obstacles, fixed or moving and has the capability of handling frequent, random, and unexpected movement of people, products, or equipment as well as items that vary in shape, weight, and fragility. The robot encompasses four elemental systems. Hands: R2's unprecedented dexterity in its hands allows it to use many of the same tools that astronauts and industry workers currently use, significantly reducing the need for specialized tools to perform multiple tasks. Arms: R2's arms are soft at multiple levels and they have redundant force sensing. R2 can safely work side-by-side with humans. Sensing and Perception: R2 shares senses similar to humans like the ability to touch and see. Interface and Control: R2 can function autonomously or it can be controlled by direct teleoperation. When functioning autonomously, R2 understands what to do and how to do it based on sensory input, carrying out tasks in real time.