Fine-tuning BERT for domain-specific keyphrase extraction from Greek medical texts, with a novel diversity ranking mechanism
The challenge of keyword extraction in a non-English biomedical domain
With the rapid growth of biomedical information, automated keyword extraction is crucial for summarizing content, topic recognition, and information retrieval. However, applying these techniques to non-English languages like Greek — especially in highly specialized domains like medicine — presents significant challenges due to the lack of dedicated, pre-processed datasets and domain-specific language models.
This project developed a robust, unsupervised machine learning pipeline that leverages contextual language representations from BERT to automatically extract highly relevant and diverse keywords from Greek biomedical texts. By fine-tuning on a custom-built biomedical corpus and implementing a novel diversity ranking mechanism, the system successfully addresses the challenge of summarizing and retrieving information within a specialized, non-English domain.
Building a novel Greek biomedical corpus from scratch
To train and evaluate the model, a novel dataset was constructed from scratch by web scraping Greek biomedical platforms (mednet.gr, sebe.gr, iatrolexi.gr). Using rule-based extraction methods built around HTML tags, a total of 1,907 medical abstracts paired with their official titles and keywords were collected. The data underwent rigorous preprocessing — including lowercasing, and the removal of tabs, HTML, and LaTeX tags via regular expressions. An additional corpus of approximately 15,000 sentences was extracted using spaCy specifically for fine-tuning the language model.
Three-stage unsupervised extraction pipeline powered by BERT
Producing high-quality keyword candidates through linguistic filtering and n-gram extraction.
Adapting Greek-BERT to biomedical semantics through Masked Language Modeling.
Dense vector representations via average pooling over BERT's last hidden state.
Maximizing relevance while ensuring the final keyword set is diverse and non-repetitive.
Partial matching metrics and qualitative diversity assessment
The system was evaluated against a test set of 192 samples using partial matching — due to the inherent difficulty of perfectly matching syntax in unsupervised extraction — and assessed through Precision, Recall, and F1 metrics at k=5 and k=10. The BERT-based approaches were benchmarked against traditional unsupervised baselines: YAKE, RAKE, and TextRank.
Beyond raw metrics, qualitative evaluation played a critical role. While graph-based methods like TextRank scored well on raw recall, they suffered from generating highly repetitive and redundant keywords. The BERT-based pipeline consistently produced semantically rich and distinctly diverse keyphrases.
BERT-based approaches heavily outperform traditional baselines
F1@5 Score — Base Greek BERT (best overall)
F1, Precision, and Recall across all models
| Model | Precision@5 | Recall@5 | F1@5 | Inference Time |
|---|---|---|---|---|
| Greek BERT (base) | 35.4% | 42.2% | 38.5% | ~1 sec |
| Bio-BERT (fine-tuned) | 34.1% | 42.4% | 37.8% | ~1 sec |
| TextRank | 28.3% | 35.6% | 31.5% | ~0.5 sec |
| YAKE! | 24.1% | 29.9% | 26.7% | ~0.3 sec |
| RAKE | 21.2% | 26.4% | 23.5% | ~0.2 sec |
Extended extraction reveals strong recall from fine-tuned model
Recall@10 — Fine-tuned Bio-BERT
At k=10, the fine-tuned Bio-BERT achieved the highest Recall of 55.5% and a competitive F1 score of 32.8%. Beyond raw metrics, qualitative evaluations showed that the BERT-powered extraction produced highly accurate, diverse keyphrases — solving the issue of repetitive keyword outputs commonly seen in graph-based algorithms like TextRank. This proved that contextual embeddings can successfully learn and summarize specialized Greek biomedical vocabularies.
Languages, frameworks, and environment
Papers used in model design and methodology