Captioning Bosch: A Twitter Bot

Abstract

The artworks by Dutch painter Hieronymus Bosch are well known for their incredible wealth of details. The popular BoschBot regularly posts small segments of the digitized paintings on Twitter, thus relieving their density and making them more accessible. CaptioningBoschBot, the Twitter bot presented in this demo, reverses the creative process of the artist: It uses the out-of-context painting segments as input for an encoder-decoder model to generate captions that interpret the painted objects. As the model was only trained on realistic, photographic images, curious interpretations of the otherworldly details can be observed. The generated captions are again posted on Twitter to encourage discussions about Bosch's masterpieces and the AI technology in general.