Kahani: Visual Storytelling

Image generation models like MidJourney, DALL-E 3, and SDXL, have made remarkable progress recently to produce visually stunning images from natural language descriptions. However, most of these models are limited by their lack of cultural awareness and diversity, and often fail to capture the subtle nuances and variations that exist in different cultures and languages.

To generate an image that matches the user’s expectations and preferences, one has to either provide extensive prompts and edit the output using sophisticated tools like Adobe Photoshop or fine-tune a model using large amounts of data and skills. Both of these approaches are time-consuming, costly, and inaccessible for most people.

Kahani: Visual Storytelling is a research prototype that allows the user to create visually striking and culturally nuanced images just by describing them in their local languages. Kahani leverages state-of-the-art techniques like Inpainting, and models like Segment Anything and GPT-4 vision to generate feedback for the candidate images.