Duplicate encoding of Chinese characters

Abstract

同一个字符拥有不同的计算机内部代码,这意味着有两个或两个以上字形在人的眼中是同一个字,而计算机却认为是不同的字。这种"人机看法不一致"会给语言信息处理带来混乱,导致信息检索不全,统计数字不准,字词分类排序不一致等情况。该文结合Unicode实例专题讨论当前计算机上存在的中文同形异码字问题,包括(a)私人造字公有化所形成的同形异码字,(b)兼容编码所形成的同形异码字,(c)建立专门的笔画部首表而形成的同形异码字,(d)半宽和全宽字形分别编码而造成的同形异码字等,并探讨解决问题的方法。 A duplicate-encoded character is a character which has been assigned two or more code points in a coding system such as Unicode.When output in distinct codes,the glyphs of a duplicate-encoded character appear the same to human users,while in the computer,they are different characters.Such a human-computer inconsistency would cause confusion in language information processing,resulting in incomplete information retrieval,inaccurate statistic calculation,and inferior quality of data sorting and categorizing.This paper discusses duplicate encoding of Chinese characters in Unicode,MS Office and the WWW,including(a)duplicate encoding arising from new code assignment in the Unihan public area to characters already encoded in the private use area,(b)duplicate encoding caused by compatibility encoding,(c)duplicate encoding brought forward by building dedicated lists for CJK strokes and radicals,and(d)duplicate encoding of characters in half-width and full-width forms.Some effective solutions to the problems are also suggested.