A Bitter Lesson for AI Researchers

If you are into Machine learning, probably you already know about the latest blog post by Prof. Rich Sutton, The Bitter Lesson. If you are not, I think you should, at least in the most fundamental level. After all, the application of AI is spanning rapidly across many scientific domains, and it is the main driver of the 4th Industrial Revolution. In this very short article, Prof. Sutton argues that when it comes to AI, researchers should abandon the idea of building human knowledge into the algorithms and focus on generic approaches that leverage the power of high-performance computing. The reason, 70 years of research shows that when it comes to building AI solutions, simpler and generic approaches (however computationally expensive) always win over more complex approaches, which try to mimic how the human mind works. In his word, "We want AI agents that can discover like we can, not which contain what we have discovered." Interested enough, that is the principle behind Deep Learning. I think that is an important lesson to keep in mind, but I leave the critics to the experts.

I recently got into AI because of my current job, and I was surprised by the variety and accuracy of algorithms for computer vision. All of them based on Deep Learning and Convolutional Neural Networks, e.g., the ImageAI python package. But I was also surprised by the development of dedicated hardware to make Machine Learning more time-efficient and mobile. One of the most recent devices is the Edge TPU, a system in a chip developed by Google which is optimized to run TensorFlow.

My current knowledge of AI is too limited to provide an educated opinion over its application in Geo-information, but maybe the experts in our group can. If you are just an enthusiast like I am; I hope at least to trigger your curiosity.