Technology that manipulates sound – from medical scans to speaker systems – has been around for a long time. But in 2016 there will be a renewed momentum behind tech that uses sound to enrich and improve the way we do things.

At Nesta, we’ll be listening out to see if this turns into a new community of startups, needs new kinds of regulation and what kind of sound tech should be centre stage at FutureFest in September.

Entertaining just got easier

The most familiar changes in sound tech in recent years have come from the entertainment sector: home recording software created a generation of bedroom DJs, from The Streets to Peep Show’s Jez. This has matured into a world of ways to create and not just consume music.

Music Tech Fest supports cutting-edge artists and technologists changing the way we make and perform music; their best-of video is well worth watching. SoundLab develops digital music technologies and approaches that help people with learning disabilities to create the music and sounds they want to make. This past year saw the first London exhibition from a brilliant deaf sound artist too.

At the same time, we are likely to see some steps forward in how we can all consume audio. The BBC's Binaural Broadcasting R&D project creates 360˚ sound experience – as if the sounds move around the listener – for any speaker arrangement including headphones. This will soon be applied to some iPlayer broadcasts.

But it’s not entertainment and media that will see the biggest audio tech changes in 2016.

Sound not screen

Sound might help provide one of the first ways to end the screen's reign as our main way of interacting with digital tech. UK startup Ultrahaptics is working with Jaguar Land Rover to create invisible air-based controls for drivers to control their settings without reaching for a button. The system uses ultrasound waves produced by an array of mini speakers to detect the specific position of a hand and create pressure on the hand as feedback when a command has been recognised.

Using similar ultrasound tech, in October British and Spanish scientists revealed a new technique for floating objects in mid-air. Borrowing from the idea of optical tweezers, the researchers programmed an array of tiny speakers to levitate, rotate and accelerate multiple small objects. In 2016 this could lead to the development of new 3D physical displays or ways of manipulating objects inside the body without cutting the skin.

Imperceptible audio waves are also at the centre of less palatable innovations. Headlines in the last couple of years have included the potential of the Amazon Fire or Hello Barbie to spy on you in your own home.

Ultrasonic waves are key to a related kind of personal data collection – cross-platform tracking. There is a crop of startups in the US that embed ultrasonic signals into TV commercials or an ad displayed in a computer browser. Nearby tablets and smartphones can detect the signal and a single user can be profiled using the multiple devices in their home. The American Centre for Democracy and Technology has filed a comment to the Federal Trade Commission, but this could make much bigger headlines in the next twelve months.

Listening tech-for-good

There is a flip side to advances in tech that listens in to our everyday activities. They make it much easier to build databases of human speech, understanding the nuances in how humans express themselves outloud. And this database will be necessary if 2016 is to be the year that acoustic evidence starts to make a difference in the visible, rather than the hidden, parts of our justice system.

Over the last half century, successful UK researchers have sat between university, commercial ventures and police services. Recent headlines include the re-analysis of hoax phone recordings of a man claiming to be the Yorkshire Ripper that delayed the original 1978 investigation. John Humble was jailed in 2005 for the ‘Wearside Jack’ calls.

One of the lead researchers, Peter French, also identified Tecwen Whittock as the man whose coughs allowed Major Charles Ingram to cheat at Who Wants to Be a Millionaire?. A BBC Frontiers radio programme from 2012 includes a range of interviews about this history.

The beginning of this decade saw some breakthrough activity using new kinds of data. 2010 also saw the first courtroom use of a UK police database of, by then, five years of recording the background hum of the National Grid. The sound waves from the grid vary slightly over time, giving off a signal of the time and place of the recording. In 2011 scientists and FBI agents in the US showed how to use a range of microphones – potentially from mobile phone and CCTV audio – to accurately recreate ballistics from crime scenes.

But to fulfil expectations of CSI-style forensic acoustics, we need to get a lot better at piecing together events from multiple low-grade recordings and understanding variation in human speech. Some work is underway to use mobile apps to create a rich database of voices. Others are researching how drug-use affects the human voice, how facewear changes how a voice sounds or how accent affects the frequency of ‘um’ or ‘er’ in someone’s speech.

The devices listening in our living rooms could be put to work creating a new database of voice recordings for researchers. That sits a lot more comfortably than advertisers snooping on our every word. Although that might take another ten years, by which time acoustic recognition could be rendered unnecessary. MIT researchers can now reconstruct a conversation from the way a nearby crisp packet moves. Perhaps by 2026 these miniscule movements will be used to identify suspects from high definition mobile video.