Please use this identifier to cite or link to this item:
http://hdl.handle.net/10397/107875
DC Field | Value | Language |
---|---|---|
dc.contributor | Department of Computing | en_US |
dc.creator | Li, X | en_US |
dc.creator | Li, J | en_US |
dc.date.accessioned | 2024-07-15T07:55:28Z | - |
dc.date.available | 2024-07-15T07:55:28Z | - |
dc.identifier.isbn | 979-8-89176-114-8 | en_US |
dc.identifier.uri | http://hdl.handle.net/10397/107875 | - |
dc.description | The 2024 Conference of the North American Chapter of the Association for Computational Linguistics, June 16-21, 2024, Mexico City | en_US |
dc.language.iso | en | en_US |
dc.publisher | Association for Computational Linguistics (ACL) | en_US |
dc.rights | © 2024 Association for Computational Linguistics | en_US |
dc.rights | Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/). | en_US |
dc.rights | The following publication Xianming Li and Jing Li. 2024. BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddings. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 792–804, Mexico City, Mexico. Association for Computational Linguistics is available at https://aclanthology.org/2024.naacl-long.45/. | en_US |
dc.title | BeLLM : backward dependency enhanced large language model for sentence embeddings | en_US |
dc.type | Conference Paper | en_US |
dc.identifier.spage | 792 | en_US |
dc.identifier.epage | 804 | en_US |
dc.identifier.volume | 1 | en_US |
dcterms.abstract | Sentence embeddings are crucial in measuring semantic similarity. Most recent studies employed large language models (LLMs) to learn sentence embeddings. Existing LLMs mainly adopted autoregressive architecture without explicit backward dependency modeling. Therefore, we examined the effects of backward dependencies in LLMs for semantic similarity measurements. Concretely, we propose a novel model: backward dependency enhanced large language model (BeLLM). It learns sentence embeddings via transforming specific attention layers from uni- to bi-directional. We extensively experiment across various semantic textual similarity (STS) tasks and downstream applications. BeLLM achieves state-of-the-art performance in varying scenarios. It shows that autoregressive LLMs benefit from backward dependencies for sentence embeddings. | en_US |
dcterms.accessRights | open access | en_US |
dcterms.bibliographicCitation | In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 792–804, Mexico City, Mexico. Association for Computational Linguistics | en_US |
dcterms.issued | 2024 | - |
dc.relation.conference | Conference of the North American Chapter of the Association for Computational Linguistics [NAACL] | en_US |
dc.description.validate | 202407 bcwh | en_US |
dc.description.oa | Version of Record | en_US |
dc.identifier.FolderNumber | a3031 | - |
dc.identifier.SubFormID | 49239 | - |
dc.description.fundingSource | RGC | en_US |
dc.description.pubStatus | Published | en_US |
dc.description.oaCategory | CC | en_US |
Appears in Collections: | Conference Paper |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
2024.naacl-long.45.pdf | 789.83 kB | Adobe PDF | View/Open |
Page views
6
Citations as of Jul 21, 2024
Downloads
1
Citations as of Jul 21, 2024
Google ScholarTM
Check
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.