PCIe 불투명 브리지 기반 HDFS 저장 방식의 성능 분석

신동렬,고병현,지민재,김성현,임승호 한국차세대컴퓨팅학회 2020 한국차세대컴퓨팅학회 논문지 Vol.16 No.6

HDFS와 같은 분산 시스템은 다수의 노드를 인터커넥트 네트워크로 연결하여 클러스터로 구성한 HPC(High Performance Computing) 시스템이다. 이러한 시스템에서 인터커넥트 네트워크의 전송방식은 전체 시스템의 성 능에 미치는 영향이 크며, 분산 시스템의 연산 및 저장 방식에 많은 영향을 미친다. 최근, HDFS 시스템은 데이터 연산 및 저장 방식의 복잡화로 네트워크 부하와 노드의 연산 부하 분석을 통한 시스템 최적화가 필요하다. 본 논문 에서는 인터커넥트 네트워크와 HDFS 시스템의 안정성 보장 기법에 대한 다양한 방법의 시스템 성능 실험을 통해 서 HDFS 시스템의 병목현상을 분석해본다. 특히, 이더넷뿐만 아니라 PCIe NTB와 같은 RDMA(Remote Direct Memory Access) 방식의 인터커넥트 네트워크 기반의 HDFS 시스템에서 Erasure Coding 기법에 대한 성능 분 석을 함으로써, RDMA 전송 가능한 PCIe NBT 기반의 인터커넥트 네트워크에서 HDFS 시스템의 안정성 향상 및 성능 향상을 위한 해결 방안을 구상하였다. A distributed system such as HDFS is a High Performance Computing(HPC) system composed of a cluster by connecting a number of nodes through an interconnect network. In such a system, the Interconnect Network has a large effect on the performance of the entire system, and has a great influence on the storage operations of distributed system. Recently, the HDFS system is required to be optimized by analyzing the network load and the computational load of nodes due to the complexity of the data operation and storage method. In this paper, we have analyzed the bottleneck of HDFS system through system performance experiment of various methods in the aspect of the stability guarantee method of HDFS system. In particular by analyzing the performance of the erasure coding technique in the RDMA interconnect network-based HDFS system such as PCIe NTB as well as Ethernet a solution for improving the stability and performance of the HDFS system in the PCIe NTB-based interconnect network capable of RDMA transfer could be figured out.

HDFS 암호화 성능 분석

박선영(Seonyoung Park),이영석(Youngseok Lee) 한국정보과학회 2014 정보과학회논문지 : 데이타베이스 Vol.41 No.1

하둡(Hadoop)은 대용량 데이터를 저장하고 처리하는 플랫폼으로 널리 사용되고 있다. 현재 하둡은 커버로스(Kerberos)를 통해 사용자 인증, HDFS(Hadoop Distributed File system) 및 맵리듀스 접근 권한 설정과 클러스터의 노드 관리 등을 제공하지만 아직 데이터를 암호화하여 저장하지 않는다. 또한 하둡은 방대한 양의 데이터를 하나의 클러스터에 저장하기 때문에 데이터 유출과 같이 기업에 막대한 피해를 초래하는 보안 사고에 매우 취약할 수 밖에 없다. 본 논문에서는 Java API에 포함되어 암호화 기능을 제공하는 JCA(Java Cryptography Architecture)를 이용하여 HDFS 암호화를 구현하고 성능 실험을 진행하였다. AES로 암호화된 HDFS의 성능 실험 결과, 암호화는 HDFS의 구조적인 한계로 한 대의 클라이언트에서 단일 쓰레드로 처리하기 때문에 25%의 성능 저하를 보였지만 복호화는 맵리듀스에 의해 HDFS에 저장된 데이터를 다수의 클러스터 노드가 처리하기 때문에 5%의 성능 저하만을 보였다. 또한 클러스터를 구성하는 노드 수를 5대에서 30대까지 늘리는 실험에서는 노드 수에 상관없이 3∼4%의 성능 저하가 나타났다. Hadoop is widely used as a platform for storing and processing large data sets. Hadoop supports Kerberos authentication and authorization for HDFS(Hadoop Distributed File system) and MapReduce, but files are not stored in an encrypted form. In addition, since Hadoop stores a huge amount of data in a cluster, data loss can cause a very serious security incident. This paper presents a more secure HDFS with encryption using JCA(Java Cryptography Architecture) and evaluates its performance. The evaluation results show a performance degradation of 25% during storing files to AES encrypted HDFS because it is processed by only a client and a single thread. In contrast, in the decryption in MapReduce, the overhead is only 5% on average since it is possible to decrypt encrypted files on multiple nodes. In addition, the results of the experiment under the different numbers of nodes show a marginal overhead of about 3∼4%.

An Efficient Replicated System for the Metadata of HDFS

Zhanye Wang,Tao Xu,Dongsheng Wang 보안공학연구지원센터 2016 International Journal of Grid and Distributed Comp Vol.9 No.5

Hadoop HDFS is an open source project from Apache Software Foundation for scalable, distributed computing and data storage. HDFS has become a critical component in today's cloud computing environment and a wide range of applications built on top of it. However, the initial design of HDFS has introduced a single-point-of-failure, since HDFS contains only one active namenode, if this namenode experiences software or hardware failures, the whole HDFS cluster is unusable, this is a reason why people are reluctant to deploy HDFS for an application whose requirement is high availability. In this paper, we present a solution to enable the high availability for HDFS's namenode through efficient metadata replication. Our solution has 3 major advantages than existing ones: We utilize multiple active namenodes, instead of one, to build a cluster to serve requests of metadata simultaneously; We implement a pub/sub system to handle the metadata replication process across these active namonodes efficiently; We also propose a novel replication algorithm to deal with the network delay when the namonodes are deployed in different areas. Based on the solution we build a prototype called NCluster and integrate it with HDFS. We evaluate NCluster to exhibit its feasibility and effectiveness. The experimental results show that our solution performs well with low replication cost, good throughput and scalability.

The Dynamic Replication Mechanism of HDFS Hot File based on Cloud Storage

Mingyong Li,Yan Ma,Meilian Chen 보안공학연구지원센터 2015 International Journal of Security and Its Applicat Vol.9 No.8

As an open source cloud storage scheme, HDFS is used by more and more large enterprises and researchers, and is actually applied to many cloud computing systems to deal with huge amounts of data. HDFS has many advantages, but there are some problems such as NameNode single point of failure, small file problem, hot issues, etc. For HDFS hot issues, this paper proposes a dynamic Replication mechanism of HDFS hot file based on cloud storage(HDFS-DRM). The mechanism includes a Replication of the dynamic adjustment mechanism and adding, deleting duplicate node selection mechanism in two parts, by increasing the NameNode, BlockMap parameters, it records the number of reading requests of each file in a certain period of time to decide whether to increase or decrease the number of copies. The mechanism presents a replica placement method based on stage historical information and node load and selects the appropriate node to add or delete copies of documents to improve the utilization efficiency of the data node storage space effectively. Experimental results show that, HDFS - DRM in hot files case, compared to native HDFS file system access latency is significantly reduced, HDFS-DRM can solve the hot issues successfully.

Adaptable I/O System based I/O Reduction for Improving the Performance of HDFS

Park, Jung Kyu,Kim, Jaeho,Koo, Sungmin,Baek, Seungjae The Institute of Electronics and Information Engin 2016 Journal of semiconductor technology and science Vol.16 No.6

In this paper, we propose a new HDFS-AIO framework to enhance HDFS with Adaptive I/O System (ADIOS), which supports many different I/O methods and enables applications to select optimal I/O routines for a particular platform without source-code modification and re-compilation. First, we customize ADIOS into a chunk-based storage system so its API semantics can fit the requirement of HDFS easily; then, we utilize Java Native Interface (JNI) to bridge HDFS and the tailored ADIOS. We use different I/O patterns to compare HDFS-AIO and the original HDFS, and the experimental results show the design feasibility and benefits. We also examine the performance of HDFS-AIO using various I/O techniques. There have been many studies that use ADIOS, however our research is expected to help in expanding the function of HDFS.

Adaptable I/O System based I/O Reduction for Improving the Performance of HDFS

Jung Kyu Park,Jaeho Kim,Sungmin Koo,Seungjae Baek 대한전자공학회 2016 Journal of semiconductor technology and science Vol.16 No.6

In this paper, we propose a new HDFS-AIO framework to enhance HDFS with Adaptive I/O System (ADIOS), which supports many different I/O methods and enables applications to select optimal I/O routines for a particular platform without source-code modification and re-compilation. First, we customize ADIOS into a chunk-based storage system so its API semantics can fit the requirement of HDFS easily; then, we utilize Java Native Interface (JNI) to bridge HDFS and the tailored ADIOS. We use different I/O patterns to compare HDFS-AIO and the original HDFS, and the experimental results show the design feasibility and benefits. We also examine the performance of HDFS-AIO using various I/O techniques. There have been many studies that use ADIOS, however our research is expected to help in expanding the function of HDFS.

Optimization Scheme for Small Files Storage Based on Hadoop Distributed File System

Yingchi Mao,Bicong Jia,Wei Min,Jiulong Wang 보안공학연구지원센터 2015 International Journal of Database Theory and Appli Vol.8 No.5

Hadoop Distributed File System (HDFS) becomes a representative cloud platform, benefiting from its reliable, scalable and low-cost storage capability. However, HDFS does not present good storage and access performance when processing a huge number of small files, because massive small files bring heavy burden on NameNode of HDFS. Meanwhile, HDFS does not provide any optimization solution for storing and accessing small files, as well as no prefetching mechanism to reduce I/O operations. This paper proposes an optimized scheme, Structured Index File Merging-SIFM, using two level file indexes, the structured metadata storage, and prefetching and caching strategy, to reduce the I/O operations and improve the access efficiency. Extensive experiments demonstrate that the proposed SIFM can effectively achieve better performance in the terms of the storing and accessing for a large number of small files on HDFS, compared with native HDFS and HAR.

LDBAS: Location-aware Data Block Allocation Strategy for HDFS-based Applications in the Cloud

( Hua Xu ),( Weiqing Liu ),( Guansheng Shu ),( Jing Li ) 한국인터넷정보학회 2018 KSII Transactions on Internet and Information Syst Vol.12 No.1

Big data processing applications have been migrated into cloud gradually, due to the advantages of cloud computing. Hadoop Distributed File System (HDFS) is one of the fundamental support systems for big data processing on MapReduce-like frameworks, such as Hadoop and Spark. Since HDFS is not aware of the co-location of virtual machines in the cloud, the default scheme of block allocation in HDFS does not fit well in the cloud environments behaving in two aspects: data reliability loss and performance degradation. In this paper, we present a novel location-aware data block allocation strategy (LDBAS). LDBAS jointly optimizes data reliability and performance for upper-layer applications by allocating data blocks according to the locations and different processing capacities of virtual nodes in the cloud. We apply LDBAS to two stages of data allocation of HDFS in the cloud (the initial data allocation and data recovery), and design the corresponding algorithms. Finally, we implement LDBAS into an actual Hadoop cluster and evaluate the performance with the benchmark suite BigDataBench. The experimental results show that LDBAS can guarantee the designed data reliability while reducing the job execution time of the I/O-intensive applications in Hadoop by 8.9% on average and up to 11.2% compared with the original Hadoop in the cloud.

A Network Load Sensitive Block Placement Strategy of HDFS

( Lingjun Meng ),( Wentao Zhao ),( Haohao Zhao ),( Yang Ding ) 한국인터넷정보학회 2015 KSII Transactions on Internet and Information Syst Vol.9 No.9

This paper investigates and analyzes the default block placement strategy of HDFS. HDFS is a typical representative distributed file system to stream vast amount of data effectively at high bandwidth to user applications. However, the default HDFS block placement policy assumes that all nodes in the cluster are homogeneous, and places blocks with a simple RoundRobin strategy without considering any nodes` resource characteristics, which decreases self-adaptability of the system. The primary contribution of this paper is the proposition of a network load sensitive block placement strategy. We have implemented our algorithm and justify it through extensive simulations and comparison with similar existing studies. The results indicate that our work not only performs much better in the data distribution but also improves write performance more significantly than the others.

Cost-effective, Large-area, Reusable Nanoimprint Molds for Polymer Nanostructures

이승우,이상민,최덕현 한국물리학회 2013 THE JOURNAL OF THE KOREAN PHYSICAL SOCIETY Vol.62 No.3

In this paper, we report the use of a heptadeca-fluoro-1,1,2,2 tetrahydrodecyl trichlorosilane (HDFS)-coated anodic aluminum oxide (AAO) template as a low-cost, large-area, reusable nanoimprint mold. Coating AAO with HDFS increases the water contact angle (CA) from 20˚ on a pristine AAO to 120˚ on an HDFS-coated AAO, thus lowering adhesion of the AAO mold. Nanostructured polydimethylsiloxane (PDMS) could then be easily obtained by using the direct imprinting method on HDFS-coated AAO without any external loads. After nanoimprinting with an HDFS-coated AAO mold, the CA of the mold rarely changes (120˚ to 115˚), thus demonstrating the possibility of a reusable AAO mold. The CA of PDMS increases up to ∼15˚ after nanoimprinting, and its transmittance decreases by 5 ∼ 30%, compared with that of a flat PDMS film, in the full visible range due to the scattering effect, which implies that the optical pathway of incident light could be extended in the nanostructured polymer. Therefore, our nanoimprinting method is expected to enable a variety of applications in next-generation soft optoelectronics.