Publications

2025

1. Under Review

   Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention

   Emily Xiao, Chin-Jou Li, Yilin Zhang, Graham Neubig, and Amanda Bertsch

   2025

   Abs

   Many-shot in-context learning has recently shown promise as an alternative to finetuning, with the major advantage that the same model can be served for multiple tasks. However, this shifts the computational burden from training-time to inference-time, making deployment of many-shot ICL challenging to justify in-practice. This cost is further increased if a custom demonstration set is retrieved for each inference example. We present Dynamic Block-Sparse Attention, a training-free framework for retrieval-based many-shot in-context learning. By combining carefully designed block-sparse attention and retrieval of cached groups of demonstrations, we achieve comparable per-example latency to finetuning while maintaining on average >95% of the best method’s accuracy across strong ICL and finetuning baselines. We hope that this will further enable the deployment of many-shot ICL at scale.

2024

1. EMBC

   Epileptic Seizure Classification with Patient-level and Video-level Contrastive Pretraining

   Chin-Jou Li, Chien-Chen Chou, Yen-Cheng Shih, Li-Chuan Kuo, Yu-Te Wang, and 4 more authors

   In 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2024

   Abs DOI

   Accurate classification of epileptic seizure types through seizure semiology analysis demands significant clinical expertise. While previous studies have employed various action recognition modules, the scarcity of labeled clinical videos has hindered the deployment of larger models. In this study, we explore unlabeled data to pretrain a transformer-based model with contrastive loss, taking advantage of the information that circumvents the need for additional annotation from medical professionals. We maximize the similarity between embeddings from the same patient and video while minimizing those from different patients and videos. Subsequently, a classification head was finetuned to distinguishing temporal lobe epilepsy (TLE) and extratemporal lobe epilepsy (exTLE), achieving a 5-fold accuracy of 0.93 and an F1 score of 0.88 on the video level (N = 57). Our results outperformed other state-of-the-art seizure classification models, demonstrating the efficacy of our approach. This suggests potential applications in clinical practice, where unlabeled data could serve as a valuable aid in improving seizure classification accuracy and patient care.

2. Interspeech Satellite

   Deep Complex U-Net with Conformer for Audio-Visual Speech Enhancement

   Shafique Ahmed, Chia-Wei Chen, WenZe Ren, Chin-Jou Li, Ernie Chu, and 5 more authors

   In 3rd COG-MHEAR Workshop on Audio-Visual Speech Enhancement (AVSEC), 2024

   Abs DOI

   Recent studies have acknowledged the advantages of incorporating visual data into SE systems. We introduce a novel audio-visual SE approach, termed DCUC-Net (deep complex U-Net with conformer network). DCUC-Net leverages complex domain features and conformer blocks. The encoder and decoder of DCUC-Net use a complex U-Net-based framework. Audio and visual signals are processed using a complex encoder and ResNet-18 model. These signals are fused using conformer blocks and transformed into enhanced speech waveforms via a complex decoder. The conformer blocks consist of self-attention mechanisms and convolutional operations, enabling DCUC-Net to capture global and local audio-visual dependencies. Our experimental results show DCUC-Net outperforms the baseline model from the COG-MHEAR AVSE Challenge 2023 by 0.14 in terms of PESQ. Additionally, DCUC-Net performs comparably to a state-of-the-art model and outperforms other models on the Taiwan Mandarin speech with video (TMSV) dataset.

3. Face swapping in seizure videos for patient deidentification

   Chin-Jou Li, Jen-Cheng Hou, Chien-Chen Chou, Yen-Cheng Shih, Stephane Dufau, and 3 more authors

   Epilepsy Research, 2024

   Abs DOI

   This study aimed to test different AI-based face-swapping models applied to videos of epileptic seizures, with the goal of protecting patient privacy while retaining clinically useful seizure semiology. We hypothesized that specific models would show differences in semiologic fidelity compared to the original clinical videos. This is the first study evaluating AI face swapping models in epileptic seizure video clips. Optimization of AI face-swapping models could enhance the accessibility of seizure videos for education and research while protecting patient privacy and maintaining semiology.

2023

1. Artificial Intelligence-Based Face Transformation in Patient Seizure Videos for Privacy Protection

   Jen-Cheng Hou, Chin-Jou Li, Chien-Chen Chou, Yen-Cheng Shih, Si-Lei Fong, and 5 more authors

   Mayo Clinic Proceedings: Digital Health, 2023

   Abs DOI

   We aim to investigate the feasibility and accuracy of artificial intelligence (AI) methods of facial deidentification in hospital-recorded epileptic seizure videos, for improved patient privacy protection while preserving clinically important features of seizure semiology.