栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

HDFS的NameNode节点信息管理

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

HDFS的NameNode节点信息管理

HDFS的NameNode信息 1、NameNode的信息存放地址

NameNode存储DataNode的元数据,NameNode主要是用于维护DataNode信息。它存储在hadoop文件夹下data/dfs/name/current/,其中以fsimage开头的文件主要存储目录、文件名称及其依赖关系,以edits开头的文件主要存储尚未来得及合并的日志记录

ls hadoop/data/dfs/name/current
#-----------------------------------------------------------------------------
edits_0000000000000000001-0000000000000000002  fsimage_0000000000000000005.md5
edits_0000000000000000003-0000000000000000003  fsimage_0000000000000000014
edits_0000000000000000004-0000000000000000005  fsimage_0000000000000000014.md5
edits_0000000000000000006-0000000000000000014  seen_txid
edits_inprogress_0000000000000000015           VERSION
fsimage_0000000000000000005
#-----------------------------------------------------------------------------
2、NameNode节点数据查看
#查看目录、文件名称及其依赖关系,将信息转化为xml格式
hdfs oiv -i hadoop_home/data/dfs/name/current/fsimage* -o save_path/filename -p XMl
#例程:
hdfs oiv -i /opt/software/hadoop313/data/dfs/name/current/fsimage_0000000000000000258 -o ~/fsimage_0000000000000000258 -p XML
#查看文件日志记录
hdfs oev -i hadoop_home/data/dfs/name/current/edits* -o save_path/filename -p XMl
#例程:
hdfs oev -i /opt/software/hadoop313/data/dfs/name/current/edits_inprogress_0000000000000000079 -o ~/edits_inprogress_0000000000000000079 -p XML
3、fsimage文件


 
  
   
    -64  
    1  
    ba631c436b806728f8ec2f54ab1e289526c90579 
    
  
   
    
    1540392763  
    
    1000  
    1006  
    0 
    
    1073741830  
    
    44 
  
  
   
    
     
      
      1  
      
      RS-6-3-1024k  
      
      1048576  
      
      DISABLED  
      
       
        
        rs  
        
        6  
        
        3 
       
      
     
      2  
      RS-3-2-1024k  
      1048576  
      DISABLED  
       
        rs  
        3  
        2 
       
      
     
      3
      RS-LEGACY-6-3-1024k
      1048576
      DISABLED
       
        rs-legacy
        6
        3
       
      
     
      4
      XOR-2-1-1024k
      1048576
      DISABLED
       
        xor
        2
        1
       
      
     
      5
      RS-10-4-1024k
      1048576
      DISABLED
       
        rs
        10
        4
       
     
    
  
  
    
    16396
    
    12
    
      
      16385
      
      DIRECTORY
      
      
      
      1640743069439
      
      root:supergroup:0755
      9223372036854775807
      -1
      
    
      16386
      DIRECTORY
      tmp
      1640738001903
      root:supergroup:0770
      -1
      -1
      
    
      16387
      DIRECTORY
      hadoop-yarn
      1640738001903
      root:supergroup:0770
      -1
      -1
      
    
      16388
      DIRECTORY
      staging
      1640738001903
      root:supergroup:0770
      -1
      -1
      
    
      16389
      DIRECTORY
      history
      1640738001940
      root:supergroup:0770
      -1
      -1
      
    
      16390
      DIRECTORY
      done
      1640738001903
      root:supergroup:0770
      -1
      -1
      
    
      16391
      DIRECTORY
      done_intermediate
      1640738001940
      root:supergroup:1777
      -1
      -1
      
    
      16392
      DIRECTORY
      test
      1640743069439
      root:supergroup:0755
      -1
      -1
      
    
      16393
      DIRECTORY
      kb16
      1640743124723
      root:supergroup:0755
      -1
      -1
      
    
      16394
      DIRECTORY
      hadoop
      1640743625760
      root:supergroup:0755
      -1
      -1
      
    
      16395
      DIRECTORY
      hive
      1640743124723
      root:supergroup:0755
      -1
      -1
     
    
    
      16396
      FILE
      
      ratings.csv
      1
      1640743625739
      1640743618694
      134217728
      root:supergroup:0644
      
      
        
          1073741825
          1001
          134217728
          
        
          1073741826
          1002
          134217728
          
        
          1073741827
          1003
          134217728
          
        
          1073741828
          1004
          134217728
          
        
          1073741829
          1005
          134217728
          
        
          1073741830
          1006
          7172347
         
       
      
      0
     
    
  
  
  
    0
    0
    
  
  
    
      16385
      16392
      16386
      
    
      16386
      16387
      
    
      16387
      16388
      
    
      16388
      16389
      
    
      16389
      16390
      16391
      
    
      16392
      16393
      
    
      16393
      16394
      16395
      
    
      16394
      16396
     
    
   
  
  
    0
    0
    0
    
    0
  
  
  
    1
    0
    0
   

4、edits文件



  
  -64
  
    OP_START_LOG_SEGMENT
    
      45
    
  

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/686021.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号