作　　者： ; ; (廖家赵）;

机构地区： 广州市地下铁道总公司

出　　处： 《计算机应用与软件》 2016年第8期76-80,共5页

摘　　要： 城轨线网小文件数据量巨大,传统的分布式文件系统很难为海量小文件存储提供符合需求的高吞吐、低延迟读写过程。根据城轨线网级业务的数据特点和以天为周期的数据访问方式,提出基于FastDFS分布式文件系统和Redis键值数据库的城轨线网海量小文件存储方法,将具有相关性的城轨小文件合并成大文件进行聚合写操作;根据FastDFS返回的大文件索引、小文件存储起始偏移量和小文件长度建立全局索引,利用Redis存储小文件名和全局索引的键值对;采用数据预取机制,预取创建时间相邻的数据。实验结果表明,相较于FastDFS系统,FastDFS-Redis系统的小文件读写吞吐量分别提高了9.35%和4.45%,达到明显改善城轨线网海量小文件的访问效率的目的。 Because of the great amount of small files for urban rail transit network system, traditional distributed file system is difficult toprovide read and write process with high throughput and low latency meeting the demand for massive small files storage. According to the datacharacteristics of urban rail transit network system and the data access method cycled in day, we propose an approach of storing massive smallfiles for urban rail transit network, which is based on FastDFS distributed file system and Redis key-value database. The method merges thesmall files with correlation for urban rail transit into a big file to accomplish the aggregated writing operation; It forms the global indexaccording to the big file indexes returned by FastDFS, the initial offset of small file storage and the lengths of small files, and uses Redis tostore key-value pair of the filename of small file and global index; It adopts data prefetching mechanism to prefech the files with adjacentcreation time. Experimental results show that compared with FastDFS distributed file system, the read throughput and write throughput ofsmall files in FastDFS-Redis storage system increase by 9. 3 5 % and 4 . 4 5 % respectively, which reaches the goal of noticeably improving thesmall file access efficiency for urban rail transit network.

关 键 词： 小文件存储 城轨线网 访问性能

领　　域： [自动化与计算机技术] [自动化与计算机技术]