• Journals
  • Discipline
  • Indexed
  • Institutions
  • About
JavaScript is disabled for your browser. Some features of this site may not work without it.
View Item 
  •   Home
  • Pontificia Universidad Católica de Chile
  • Onomázein: Revista de Linguística, Filología y Traducción
  • View Item
  •   Home
  • Pontificia Universidad Católica de Chile
  • Onomázein: Revista de Linguística, Filología y Traducción
  • View Item

An intelligent extension of the training set for the Persian n-gram language model: an enrichment algorithm

An intelligent extension of the training set for the Persian n-gram language model: an enrichment algorithm

Author
Motavallian, Rezvan

Komeily, Masoud

Full text
https://onomazein.letras.uc.cl/index.php/onom/article/view/69745
10.7764/onomazein.61.09
Abstract
In this article, we are going to introduce an automatic mechanism to intelligently extend the training set to improve the n-gram language model of Persian. Given the free word-order property in Persian, our enrichment algorithm diversifies n-gram combinations in baseline training data through dependency reordering, adding permissible sentences and filtering ungrammatical sentences using a hybrid empirical (heuristic) and linguistic approach. Experiments performed on baseline training set (taken from a standard Persian corpus) and the resulting enriched training set indicate a declining trend in average relative perplexity (between 34% to 73%) for informal/spoken vs. formal/written Persian test data.
Metadata
Show full item record
Discipline
Artes, Arquitectura y UrbanismoCiencias Agrarias, Forestales y VeterinariasCiencias Exactas y NaturalesCiencias SocialesDerechoEconomía y AdministraciónFilosofía y HumanidadesIngenieríaMedicinaMultidisciplinarias
Institutions
Universidad de ChileUniversidad Católica de ChileUniversidad de Santiago de ChileUniversidad de ConcepciónUniversidad Austral de ChileUniversidad Católica de ValparaísoUniversidad del Bio BioUniversidad de ValparaísoUniversidad Católica del Nortemore

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

LoginRegister
Dirección de Servicios de Información y Bibliotecas (SISIB) - Universidad de Chile
© 2019 Dspace - Modificado por SISIB