Please read this carefully (or your computer may understand it better than you do…) “Will I be replaced by a computer?”. This question was not the only thing on my mind when I attended the ACL annual conference – but I’d be lying if I said it didn’t cross it.
Everyone’s familiar with the idea that, given enough time, a monkey bashing away at the keyboard will eventually type out Hamlet. But what would it take for your computer to actually understand Hamlet? Natural Language Processing (or NLP for short) is all about developing computer algorithms that can understand texts that we human beings have written.
It’s a booming field of computer science at the moment. This is mostly because of the huge amount of information we’re inputting onto computers through the posts of Twitter, Facebook and corporate documents. Wading through it all to make some sense of it would take humans an unfeasibly long time, but now we’re starting to get computers to do it at lightning speed. They can even translate it and apply ‘sentiment classification’ to textual content according to the various opinions expressed. And although it’s been with us for over five decades, NLP (also known as Text Analytics and Computational Linguistics) is gaining a lot of attention right now because of the amount of available textual data and business interest.
I first attended the Association for Computational Linguistics (ACL)’s annual congress 12 years ago, when most of the people there were from universities. Then around five years ago the likes of Google, IBM and Amazon (known by many of those academics as ‘the dark side’!) started to show up. But at this year’s conference in Berlin, attended by over 1600 people, I noticed the make-up was from a much wider range of backgrounds: around half of them affiliated to small and mid-size enterprises (including three Black Swan data scientists!) – NLP research has been hit by the start-up era!
ACL is the best place to keep up to date with the latest developments in NLP research. This year, there were seven parallel sessions running, consisting of presentations by top scientists from Stanford University, MIT, Cambridge University, Google, and Microsoft. While the topics were diverse, it wasn’t hard to detect a central issue running through all of them: around a quarter of presentations focused on new deep architectures for NLP tasks; while another quarter used deep learning techniques in their solutions. Deep learning is clearly big this year! It came out of machine learning around 10 years ago and has been the architect of revolutionary success stories in speech technology and image processing.
However, it remains to be seen whether deep learning can achieve such an important breakthrough in NLP – signal processing of audio and images is one thing, but for computers to understand texts is a much more complex task.
A Force for Good – Or Evil?!
There’s a massive commitment to finding advances in deep learning, since clearly the opportunities are huge too. Whilst chatting during the breaks, I realised that some of the people in this field have already dedicated a decade or more of their lives to deep learning in NLP – while another group are still waiting for these people to deliver tools they can make use of! Then, for balance, there were those I spoke to who think deep learning is just plain evil. One big fear, of course, is that it might even put people like us out of work! Why pay someone a wage to do something you can get a computer to do for free?
Deep learning’s big promise is precisely that: to find the best architecture for a certain task instead of needing a human to engineer that. A professor told me that he recently decided to buy more CPU/GPU instead of hiring an engineer. I’m also an NLP engineer but I’m not afraid that a computer will replace me in the next decade.
Deep learning can definitely contribute if you focus on a specific sub-problem for years, much in the way that academics do. On the other hand, if you are regularly challenged with new problems, like we are at Black Swan, the creativity that human engineers provide in these situations cannot be replaced by machines.
So the good news is that my attending ACL has not given me sleepless nights. In fact not only was it a great event packed to the brim with stimulating talks to spark fresh thinking, on a personal level I found it very comforting to see that we’re clearly bang on track at Black Swan, creating state-of-the-art research – in fact I couldn’t help but notice how similar in approach one of the papers presented was to the geolocation system we’d developed during our Szeged hackathon just three weeks before!