Cross-Lingual Information Retrieval Using NLP

Abstract

Cross-Lingual Information Retrieval (CLIR) playsapivotal role in bridg inglinguistic gaps with in digital document management repositories.Traditionalinformationretrievalmodelsrelystrictlyonmono-lingualmatching, frequently failing whenauser submitsaquery inalanguagedistinctfromthe sourcedocumentation language.This paperpresentsaweb-based,highlyefficientCLIRframeworkengineeredspecificallyforEnglish and Kannada document processing. Built on top of a Flaskbackend,thesystemingestsunstructuredmultipage PDF documents, extractslocalizedtextualinformationvia Py Mu PDF, and implements language detection work flows. When cross-lingualvarianceisencountered,thesysteminvokesadedicatedEnglish-Kannadamachinetranslationwrapper.DocumentcomponentsareindexedusingTermFrequencyInverseDocumentFrequency(TFIDF)vectorsandmatchedagainsttheexpandedqueryusingCosineSimilarityheuristics.Experimentalevaluationsdemonstratehighprecisioninsourcebasedanswerextractionandexactpagelevelmapping.Theframeworkfeaturesanadditional,dedicatedautomatedquestion-papersolvercapabilityusinggranularregex pattern-matching routines.

Introduction

The research proposes a Cross-Lingual Information Retrieval (CLIR) system that enables users to search English-language PDF documents using queries in either English or Kannada. It addresses the limitations of traditional keyword-based search, which often fails when the query language differs from the language of the document.

1. Introduction and Objectives

Academic institutions and enterprises store large amounts of information in PDF documents. Conventional search engines rely on matching exact keywords, making it difficult to retrieve relevant information when users search in a regional language such as Kannada.

The proposed system combines PDF text extraction, language detection, machine translation, and similarity-based retrieval to provide:

• Language-independent document searching.

• Accurate extraction of relevant information with exact source page references.

• Processing and analysis of multiple-choice question papers.

• Confidence scores and document citations to help users verify results.

2. Literature Survey

The research builds on previous work in cross-lingual information retrieval, semantic embeddings, machine translation, and deep learning. Earlier studies suggest that translation, query expansion, contextual text representations, and similarity calculations can improve retrieval across languages.

These approaches provide the foundation for a system that connects Kannada queries with relevant content in English documents.

3. Methodology and System Architecture

The system follows a modular processing pipeline:

1. PDF upload and text extraction: PyMuPDF extracts text from PDF files while preserving source-page metadata.

2. Language detection: The system identifies whether the user's query is written in English or Kannada.

3. Text preprocessing and translation: The query is prepared for searching, with conditional translation used to bridge language differences.

4. Text chunking: Extracted document content is divided into smaller segments for efficient retrieval.

5. TF-IDF vectorization: Text is represented numerically according to the importance of terms within individual segments and the document collection.

6. Cosine similarity: Query and document vectors are compared to identify relevant passages.

7. Relevance ranking: Retrieved segments are ordered according to their similarity to the query.

8. Answer extraction: Relevant information is extracted from the selected passages.

9. Source mapping: Results are linked to their original document pages to support verification.

4. Implementation

The architecture integrates PDF processing, language recognition, translation, text preprocessing, vector representation, similarity matching, relevance ranking, answer extraction, and source-page mapping.

PyMuPDF supports document extraction, while TF-IDF and cosine similarity provide the core retrieval mechanism. Translation enables cross-language searches, and page metadata connects retrieved information to its original source.

5. Experimental Results and Discussion

The reported tests indicate that the system can retrieve information from English technical documents using Kannada queries. The translation component reportedly preserves important domain-specific meaning, while source-page tracking helps users verify extracted answers.

The system also provides confidence percentages, document citations, and exact page numbers to improve transparency.

However, the claim that the system completely prevents hallucinations should be interpreted cautiously. Page-level citations improve traceability, but they do not guarantee that every extracted answer is correct. The text also does not provide detailed quantitative metrics, benchmark comparisons, or numerical accuracy results to independently establish the system's performance.

Conclusion

Thispaperestablishesareliable,productionreadyCLIRarchitecturetailoredforEnglishandKannadadocumentexploration.BycleanlycombiningPyMuPDFparsers,languagedetectionmatrices,andTFIDFmatchingalgorithms,thesystembridgescommunicationgapsacrosslocalregionallanguages.Futureimprovementswillfocusonmigratingthevectormatchingenginetodensesemantictransformerrepresentations(suchasmultilingualminiLMlayers)andextending processing capabilities to handle scanned image assets via OCR pipelines.

References

[1] Y. A.Mohamed et al., “The Impact of Artificial Intelligence on Language Translation: A Review,” IEEE Access, 2024. [2] M.Kalantari,M.Feghhi,andN.Mozayani,“BenchmarkingEmbeddingModelsforPersian-LanguageSemantic Information Retrieval,” IEEE IKT, 2024. [3] “Information Retrieval with Query Expansion and Re-ranking: A Survey,” IET Conference Publication, 2024/2025. [4] “OptimizingCrossLanguageInformationRetrievalandContextTranslationUsingBERTModelandDeepLearning Techniques,” IEEE Conference Publication, 2025. [5] D.Kotharietal.,“Cross-LingualTransferLearningforMedicalNERandQuestionAnsweringinLow-Resource Language (Hindi),” IEEE MISSF, 2026.

Copyright

Copyright © 2026 Sanjana C, Sushma S Kachavi, Vandana K S, Moulyashree M P. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.