Getting up close and personal with a computer: How smart textiles aid communication with virtual worlds

Say goodbye to games controllers or keyboards, say hello to highly flexible, ultrathin silicone films. Professor Stefan Seelecke and his team at Saarland University are currently developing a novel type of technology that allows humans and computers to communicate more naturally and more intuitively.

When their thin polymer film is integrated into an assembly operator’s glove or deposited onto a display screen, the film functions as an interactive mediator that can tell a computer system what the human operator wants while also providing tactile feedback to the user in the form of pulses, vibrations or taps, or audible feedback in the form of acoustic signals. The research team will be exhibiting their multifunctional technology at this year’s Hannover Messe from 17 April to 21 April (Hall 2, Stand B34).

Whether we’re using a controller, a keyboard or a mouse, when we want to communicate with a computer we usually press buttons or keys. That restricts how we communicate in virtual environments and tends to make the interaction less natural than direct communication between two people. When we interact with computers, we focus on finding the right button or key on our data input device. Computers are normally unable to understand our gestures or finger movements unless special sensors, cameras or other motion-tracking tech is added.

But the new technology developed by Stefan Seelecke and his team doesn’t need additional tools, sensors or cameras—all it uses is a simple polymer film that effectively provides the computer with a new sense organ. The film essentially acts as a medium for spontaneous interaction and communication between a human user and a machine.

“The films that we use are only about 50 microns thick (1 micron = 1 thousandth of a millimeter), which makes them both very thin and extremely light. The films are essentially ready to use and don’t require any additional technology to make them into flexible and elastically deformable sensors and/or actuators—they are, effectively, a kind of micromotor,” explained Stefan Seelecke, Professor of Intelligent Material Systems at Saarland University, who together with his research team is studying these smart polymer films at ZeMA (Center for Mechatronics and Automation Technology) in Saarbrücken.

What that means in practice is that if the film is applied to a textile surface or other object, it can begin supplying the computer with information. And it provides feedback to the user in the form of haptic signals (pulses, vibrations or pushing motions) or acoustic signals (sounds).

The range of possible applications of these smart textiles or surfaces is immense. For example, a gaming glove or an industrial glove fitted with the film would enable the wearer to communicate virtually using their hand—one of the most important communication tools used by humans. Giacomo Morretti and Sebastian Gratz-Kelly, two members of Seelecke’s research team, have fitted an industrial glove with the film and the glove can now tell the computer system how the operator is moving their hand and fingers.

In a virtual ‘Industry 4.0’ environment, the smart glove could use gesture recognition to assist the wearer when selecting a component to install or could facilitate grip strength measurements that could help the operator when tightening bolts. The glove could also use acoustic signals to warn the wearer if a mistake has been made during the assembly process.

The operator would also be able to control different processes simply by gesturing or moving their hand. It doesn’t require a lot of fantasy to see that combining smart gloves and textiles with a VR headset would make interacting in virtual games and work environments far more intuitive and lifelike than when using a games controller.

Another promising area of application involves incorporating the film into clothing that children in hospital isolation units can wear to experience bodily contact with their parents. A pullover fitted with the smart film could act as a second skin that would transmit hugs and strokes to the child when mum or dad stroke or caress a second smart textile.

“We want to give children in hospital isolation units and their parents the opportunity to meet and interact in a safe virtual space. Our aim is to create a lifelike and emotionally immersive environment in which the child and the parents can interact virtually using vision, hearing and, importantly, touch,” said Martina Lehser.

“From an engineering perspective, the film is a dielectric elastomer that allows us to combine sensor, actuator and acoustic functionalities,” explained Paul Motzki who holds a cross-institutional professorship in smart material systems for innovative production at Saarland University and at ZeMA, where he heads the research area ‘Smart Material Systems.’

The silicone polymer film is printed on both sides with an electrically conductive material that responds to changing electric fields while also exhibiting extremely low power consumption. If a voltage is applied to the film, the resulting electrostatic attractive forces cause the film to compress.

“When it compresses, the film extends laterally thus increasing its surface area, which in turn alters the electrical capacitance of the film,” explained Motzki. “We can assign a precise electrical capacitance value to any particular position of the film.”

This imparts sensor properties to the film without the need for additional technology. As the hand and fingers move inside the glove, the film deforms, getting stretched, pulled or compressed, which generates a large sequence of individual measurement values. “We have developed intelligent algorithms that allow these motion sequences to be quantified and subsequently processed in a computer,” explained Sebastian Gratz-Kelly, who is currently studying smart textiles as part of his doctoral research project.

The algorithms also allow the research team to precisely control the motion of the silicone film. “The motions we can create range from high-frequency vibrations down to slow pulsing or flexing motions and we can also hold the film in a specific fixed position. We can control the type and frequency of the vibrations very precisely,” said Sophie Nalbach.

The team can therefore control the film so that it deforms and pushes against the user’s finger. Rapid film flexing underneath the user’s fingertip can generate the feeling of a tiny, raised slider button or create a slight feeling of pressure such as when someone turns on an actual switch. The film can also generate individual acoustic tones or even multiple tones if several vibrational frequencies are superimposed on one another.

The research team will be at this year’s Hannover Messe where they will be showcasing a number of prototypes of their smart textiles and haptic displays, including the intelligent industrial glove.