1. Accent Adaptation in ASR Systems.
Proposer – Prof. Preethi Jyothi, IIT Bombay
Abstract: Voice-driven automated agents such as personal assistants are becoming increasingly popular. However, in a multi-lingual and multi-cultural country like India, deploying such agents to engage with large sections of the population is highly challenging. A major hindrance in this regard is the difficulty the agents would face in understanding varying speech accents of the users. Even when the language of interaction with the underlying automatic speech recognition (ASR) system is restricted to a lingua franca (such as English), the accent of the speaker can vary dramatically based on their cultural and linguistic background, posing a fundamental challenge for ASR systems. Tackling this challenge will be a necessary first step towards building socially accepted and commercially successful agents in the Indian context.
The main focus of this project will be to take this first step, by improving state-of-the-art performance of ASR systems on accented speech – specifically, speech with Indian accents. We shall develop deep neural network based acoustic models that will be trained using not only accented speech data but also speech in the native languages associated with the accent. We shall also develop a tool that will be trained to identify various Indian accents automatically. Finally, we shall investigate how accented-speech-ASR can be effectively incorporated into intelligent agents to help them act in socio-culturally appropriate ways.
2. HollyChat: Domain Specific Conversation Systems.
Proposer – Prof. Mitesh Khapra, IIT Madras
Abstract: Most of the AI systems today are driven by three key components (i) data (ii) common sense knowledge and (iii) powerful learning algorithms which can harness this data and knowledge to learn task specific meaningful patterns. Recently there has been a lot of interest in domain-specific dialog systems with several downstream use cases such as shopping assistants, customer support, tour guides, etc. Most of the existing dialog systems are partly in line with the trend mentioned above – in the sense that they are data driven and use powerful algorithms (deep recurrent neural networks and their variants). However, we are nowhere close to building deployable domain-specific conversation systems. One of the primary reasons for this shortfall is that such agents do not exploit any common sense or real-world knowledge, and thereby are unable to maintain a richer context of the conversation. We propose to focus on domain specific conversation systems which use domain specific knowledge graphs as external memory. The idea is to use a domain-specific knowledge graph to discover the latent intent of the user and keep the conversation coherent with this intent. For example, the knowledge graph driven intention network could map the user’s utterance \textit{“I really liked the action sequences in Inception (movie)”} to all tuples containing the entity \textit{“Inception”} and keep the conversation restricted to concepts linked to this entity. An appropriate response in this case could be \textit{“Yes, indeed, movies directed by Christopher Nolan are known for their action”} which contains entities and predicates linked to \textit{“Inception”}. This would help in the task of dialog planning and also address the problem of keeping track of large contexts (which would be required for longer conversations containing many turns). In this case, the model could learn to abstract out the context in terms of entities and predicates in the knowledge base and just track these and their immediate neighborhood in the knowledge graph.
3. Detection of Aggressive Behavior on Social Media.
Proposer: Prof. Ritesh Kumar, Ambedkar University
Abstract: As the interaction over the web has increased, incidents of aggression and related events like trolling, cyberbullying, flaming, hate speech, etc. too have increased manifold across the globe. While most of these behaviour like bullying or hate speech have predated the Internet, the reach and extent of the Internet has given these an unprecedented power and influence to affect the lives of billions of people. It has been observed that the incidents of aggression and unratified verbal behaviour has not remained just a minor nuisance but has acquired the form of a major criminal activity that affects a large number of people. These have not only created mental and psychological agony to the users of the web but has in fact forced people to deactivate their accounts and in rare instances also commit suicides. So it is of utmost significance and importance that some preventive measures be taken to safeguard the interests of the people using the web as well as of the web itself such that it remains a viable medium of communication and connection, in general.
The aim of the project is to develop a prototype that could automatically tell ratified (both aggressive as well as non-aggressive) linguistic behaviour from unratified (aggressive) ones (recognised by varied names like flaming, aggression, trolling, hate speech, cyberbullying, etc.) on the online forums (especially social media and news/opinion websites/blogs). I propose to develop the system using supervised text classification methods combined with sequence models that would be trained using a dataset annotated with labels pertaining to aggression in Hindi and Hindi-English code-mixed data collected from different kinds of Facebook Pages including those of news/media organisations, support/help groups, celebrity pages and other similar pages as well as from certain focussed topics/themes on Twitter.
4. Utilising Social Media for Disaster Relief: Linguistic Analysis of Resource Requests and Offers on Twitter.
Proposer: Prof. Saptarshi Ghosh, IIT Kharagpur
Abstract: Effective coordination of post-disaster relief operations depends critically on the availability of reliable situational information, as well as on citizen participation in the operations. The advent of online social media (e.g., Twitter, Facebook) and the widespread availability of mobile Internet today enable regular citizens to contribute to the relief operations, even if they are themselves stuck in the disaster effected zones. The aim of this project is to develop mechanisms for utilizing online social media for helping post-disaster relief operations. Specifically, our goal is to develop tools that help coordinate resource requests and resource offerings, to ensure optimal resource utilization during the disaster.
To this end, we first propose to analyze the linguistic characteristics of resource requests and resource offerings posted on Twitter during various disaster scenarios. This analysis is likely to yield insights into how different people phrase requests and offers for resources, in various languages. Next, we plan to utilize the insights obtained from the linguistic analysis, to build systems that will help coordinate the resource requests and offerings. Specifically, we envision building an automated bot that responds appropriately to resource requests and offers, and then matches corresponding requests and offers.