Deep Neural Network Learns to Judge Books by Their Covers

A machine-vision algorithm can tell a book’s genre by looking at its cover. This paves the way for ‘AI’ systems to design the covers themselves.

The idiom “never judge a book by its cover” warns against evaluating something purely by the way it looks. And yet book covers are designed to give readers an idea of the content, to make them want to pick up a book and read it.

And humans are quite good at it. It is relatively straightforward to pick out a cookery book or a biography or a travel guide just by looking at the cover.

And that raises an interesting question: can machines judge books by their covers, too? We already know they judge people by their faces.

Today we get an answer thanks to the work of Brian Kenji Iwana and Seiichi Uchida at Kyushu University in Japan. They have trained a deep neural network to study book covers and determine the category of book they come from, Technology Review website reported.

Their method is straightforward. Iwana and Uchida downloaded 137,788 unique book covers from Amazon.com along with the genre of book. There are 20 possible genres but where a book was listed in more than one category, the researchers used just the first.

The algorithm listed the correct genre in its top 3 choices over 40% of the time and found the exact genre more than 20% of the time. “This shows that classification of book cover designs is possible, although a very difficult task,” say Iwana and Uchida.

Some categories turn out to be easier to recognize than others. For example, travel books and books about computer and technology are relatively easy for the neural network to spot because book designers consistently use similar images and design for these genres.

The algorithm also confused children’s books with comics and graphic novels as well as medical books and science books. Perhaps that is also understandable given the similarities between these categories.