无聊尝试一下golang连接hdfs, 写了个hello world
golang包地址
https://github.com/vladimirvivien/gowfs
安装
go get github.com/vladimirvivien/gowfs
写程序之前需要修改hadoop的两个配置文件
分别是
hsdfs-site.xml 里的
dfs.webhdfs.enabled
<property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property>
core-site.xml 里的
hadoop.http.staticuser.user
<property> <name>hadoop.http.staticuser.user</name> <value>hadoop</value> </property>
简单的例子
package main
import (
"fmt"
"log"
"github.com/vladimirvivien/gowfs"
)
func main() {
fs, err := gowfs.NewFileSystem(gowfs.Configuration{Addr: "localhost:50070", User: "hadoop"})
if err != nil{
log.Fatal(err)
}
checksum, err := fs.GetFileChecksum(gowfs.Path{Name: "/user/kafka/hello.txt"})
if err != nil {
log.Fatal(err)
}
fmt.Println (checksum)
createTestDir(fs, "/user/kafka/hello1.txt")
}
func createTestDir(fs *gowfs.FileSystem, hdfsPath string) {
path := gowfs.Path{Name:hdfsPath}
ok, err := fs.MkDirs(path, 0744)
if err != nil || !ok {
log.Fatal("Unable to create test directory ", hdfsPath, ":", err)
}
log.Println ("HDFS Path ", path.Name, " created.")
}
有疑问加站长微信联系（非本文作者）
