Research Interests
Interpretable and Controllable Language Models
- Language Acquisition from Limited Sources
- Language models shape the world with given data. They work well with quality and abundant data while providing biased perspectives with scarce, poor data. However, quality data is often expensive to collect or private to get access. This challenge has inspired me to study how to encourage language models to acquire language proficiency and reasoning ability from a limited quantity or quality of resources.
- Causal Inference, not Correlation
- The heavy reliance on correlations between input observation and output predictions limits language models to reason about cause-and-effect relationships that are not explicitly present in the text. High correlations do not always uncover the causality of events. Causal inference is a crucial topic to control language models against biased data.
- Information Quantification
- Computation creates valuable information. The usable information for models varies depending on their interaction between input and output resources. Exploring how to quantify information is essential for interpreting model behaviors.
- Emergent Knowledge of Language Models
- Language models emerge new knowledge that is not present in smaller models when the scale grows. The emergent knowledge is a valuable yet underexplored resource. This has motivated me to investigate them regarding the model capabilities of how trustworthy they are for transparent usage in real-world scenarios.