Neuron SoundWare: Making Sense of Sound

Neuron Soundware is what we often call a “deep tech” startup. Like several of StartupYard’s alumni companies, they are unique in that they are operating at the edge of current technology, and developing processes and software that have never been tried before.

Neuron Soundware’s mission is bring machine learning and neural networks to sound and voice analysis- creating interfaces between machines and people, and between machines and mechanical devices, that are highly intelligent and adaptable to many applications. I caught up with CoFounder and CEO Pavel Konecny this week to talk about the company and its mission to understand and process sound:

Hi Pavel, tell us a bit about Neuron Soundware, what you do, and how the company came to be.

We build and train software that understands audio. Our technology is practically an auditory cortex in a computer- a digital brain. It’s all based on deep learning – the fastest-growing field in machine learning. That’s the kind of technology, that has just recently beaten the best human player in Go. It is all enabled by the increasing power of computing – especially thanks to GPU advances.

We started working on some prototypes in October last year. I worked for a global IT firm for the past 12 years, and I just returned to Prague after spending over 3 years in the Sydney office. It was a great time, but I felt that I needed to follow my passion for AI. So I convinced my friend and high school classmate. We initially worked on some of my ideas around data compression using neural networks. In November, we applied for StartupYard. It was a good decision, and it helped us to find the right business field for our technology.

Your team is probably working on the most complex problems of any startup in our program right now. How is your team uniquely qualified to solve problems using machine learning and neural networks?

All of the co-founders have a strong technical background. I studied cybernetics and biomedical science. And I’ve worked on many projects with different technologies. I did the first Hadoop Big Data project in Australia, and implemented high-performance calculations for smart meters. We were about to digitalize a video archive, so I also built a pilot implementation for the facial recognition of cricket players last year. That was a very exciting experience.

Filip Sedlak has a Master’s degree in chemistry informatics. He worked many years for pharmaceutical companies. He was in charge of taking an algorithm and building applications for researchers in genetics. He is really brilliant at that. I am very glad that he joined our team.

 

Pavel Klinger is my very good friend. He studied biophysics at Charles University in Prague. He is an excellent coder as he has been programming since his childhood. He is also a musician. So that is one of the reasons we focus on audio. Neural networks are his old hobby. He built his first one when he was at high school. He wanted something to write his school homework instead of him. It did not work that well, but he could type a bit faster as the neural net was predicting the word he was going to type based on the first few letters. So it was an easy task to convince him to start a company using neural networks.

Can you tell us a bit more about how neural networks and machine learning software can help manufacturers and other industries?

Our technology analyzes sounds that machinery makes. We train it to recognize any anomalies or known issues. Sound and vibration are the most efficient way to detect mechanical malfunctions. However, the moving parts acoustics were typically too complex to make it practical. Till now. Our technology can manage these complexities, so we can inspect enclosed parts like gearbox using the sound it makes.

Our software is sort of like a very streamlined, simplified mechanic- but one that never gets distracted or bored. It can learn how an engine normal sounds, and it can learn which sounds signal a mechanical problem.

Neuron Soundware’s technology works just like a human brain- but because of the complexity of a real brain, ours is only about as powerful as the brain of a small bee. But that’s enough, really, to understand quite a bit about complex sounds, and recognize problems by listening to them in a huge range of applications.

Neuron Soundware technology can also learn from “training data,” to reconstruct sounds and even voices, based on what it has already listened to. This has enormous applications in the future, when it comes to anything from call centers, to natural language AIs.

Our technology can eventually be used to generate the voice of a completely artificial mind, with emotion, ability to listen, and detect and understand complex emotions in whomever it’s talking to. It’s difficult to overstate how many applications this technology will have in the next decade. It’s simply enormous. 

For now, we are already able to reconstruct a real voice that the software has listened to, and to reproduce speech without using any sound files. Soon, we will be able to generate original, unique speech as well. I plan to demo this technology on April 6th at the StartupYard 2016 Demo Day.

 

What are some of the other possible applications for your machine learning technology?

In addition to the industry sounds, we are developing some advanced services for call centers. It sounds like magic, but we are getting close to the production stage. For now. we are working on emotion detection and comprehensive voice modulation. 

The way you say something is often more important than what you say. So we can start by modifying a customer service agent’s voice perfectly to the emotional state of the customer. For example, the direct sales team would sound nicer and more trustable. We can change accents, and make many other changes. In time, our technology could completely replace call center agents, with machines that never make mistakes, and never get tired or frustrated.
We also have plans in the area of IoT devices, where we could provide diagnostics and sound recognition in conjunction with other services. We’re currently working on a potential partnership in this area with a major Czech printing and IoT company, who have been incredibly supportive so far.

Q: You’re focusing initially on partnerships with device companies, such as 3D printer manufacturers. What do you hope to accomplish business-wise in the next year?

Industry sounds are quite simple compared to emotional detection in voices. We have achieved extremely high accuracy. Hence, we are planning to first address manufacturers with high-value products and large high quantities, such as car makers.

Neuron Soundware technology could provide a sound “guardian” to any moving part. We could hear that something is broken or even predict that it is going to break. We would license our algorithms and charge fees for every unit equipped with our system.

How about the longer-term strategy? Where do you hope to be as a company in 3-5 years? What kind of things will Neuron Soundware technology be doing further in the future?

Our plan is to master the process of learning. We believe that the next generation of applications will not be coded but “trained.” 

For example, we would replace most of the people working in call centers with programs that learn on the job. We would take a few months of historical data from the call center (voice and screen recording) and process it. Our algorithms would develop a complex customer interaction model that would not only consist of the conversation with the customer but also management the customer data in the backend systems. Call center agents are a kind of interface between calling customer and company systems. With our service in place, the call center traffic will be managed with a fraction of the current staff.

You’re a bit of a futurist, I’ve noticed. How do you think the overall role of AI and machine learning will grow within the foreseeable future?

A lot of our jobs will be done by machines and automated services. For example, it is much harder to make a living as a translator these days, when you are competing with Google translate. You might think that people are still better, but they won’t be forever. The same would happen in a lot of other areas.

Do you want to sort the best photos, videos and create an appealing movie from your vacation? Done. Exploration of what is the best market for my product would be just one click? An automated tool will create a set of ads and test them via Facebook in a day.

It would be a huge impact on the society. New jobs would need to be invented. We might even see taxation of computation time. If Moore’s law stays valid a few more decades, we would all have a choice. Either die or move our minds into machines and then live forever. But that’s all still in the far future. 

For now, AI still needs a way to interact with people and objects.

Neuron Soundware is paving the way for machines to be able to interact with the world based on sound- which is an incredibly important sense for a machine to have. Much of our conception of AI today is about natural language processing- programs listening to and understanding us and their environment. But that requires many innovations that are still in the future. We want to lead that effort.

Of any team at StartupYard, Neuron Soundware has had to work hardest on finding a go-to-market strategy. Why do you think that’s so difficult to do with AI and machine learning?

That is the counter-intuitiveness of machine learning. You do not need a lot of coding, but you need good training data sets. We can not build our services without our clients. We initially wanted to focus only on the call centers.  However, the device diagnostics using sound is the low hanging fruit.

As we say, a machine doesn’t have a lawyer! It’s ethically complicated to work with live customers, and that is a part of the legal and business world that has to keep evolving in the next few years. As you can see from many other areas of advancement in AI, such as self-driving cars, new laws and regulations are necessary, and they are coming. We believe the benefits to people far outweigh the short-term drawbacks.

You’ve been working on several potential partnerships. Can you tell us more about those?

We do have a partnership memorandum with some Czech universities. That would help our company in different dimensions. Firstly, we would have access to the latest research and people in our key areas. That would help us to grow our team and knowledge. Secondly, we can fund our product development via cooperation with the university laboratories as it is close to the leading edge of scientific research. So we could leverage their server infrastructure as it will be probably our highest cost. We are also looking for a strong business partner.

How has the StartupYard program affected the development of Neuron Software? Where do you think you would be if you hadn’t joined the accelerator?

It was a really great experience. We gained a lot of contacts, ideas and quick feedback. The mentors from StartupYard are valuable providing support to the Neuron Soundware team. We changed our focus from music to voice and industry sounds, largely due to the input of mentors we met in the program. A long series of workshops gave us an overview of what to expect and master in so many different areas. 

I would personally welcome prolonging the program by 3-4 weeks. So we would have a bit more time to digest all that information, meet more customers and iterate even more on our product.

Has any particular mentor at StartupYard had a particularly strong impact on your company’s development? How so?

Well I think it might be the StartupYard team, You and Cedric Maloux. All of the mentors gave us a lot of valuable input, but the StartupYard team also tells us what *not* to do. And that’s often even more important. So when I’m in doubt, I ask myself: “would Lloyd or Cedric do this?” And that helps me to find the right path.