golang开源分布式文件系统weed-fs的安装与运用

qq286210662 · · 10443 次点击 · · 开始浏览    
这是一个创建于 的文章,其中的信息可能已经有所发展或是发生改变。

1.安装配置golang开发环境

       GoGoogle开发的一种编译型,可平行化,并具有垃圾回收功能的编程语言,本文写作时,go的最新版本是1.4.2,可到http://www.golangtc.com/download去下载go1.4.2.linux-amd64.tar.gz,下载后将压缩包上传并解压到指定的目录下(例:/tmp/chengxf/go)。

        配置环境变量,编辑/etc/profile文件,添加如下内容:

        GOHOME=/tmp/chengxf
        export GOROOT=$GOHOME/go
        export GOPATH=$GOHOME/gopath
        export PATH=$PATH:$GOHOME/go/bin
        export PATH=$PATH:$GOPATH/bin

        保存退出,执行source /etc/profile重新加载环境变量后,执行go version,若能正常显示go的版本信息,则配置成功。


2.下载并安装weed-fs

        weed-fs又称为seaweed-fs,是一个简单且高性能的分布式存储系统,他有如下功能:

        a. 存储海量文件

        b. 能快速访问存储的文件

        参考weed-fs的官网https://github.com/chrislusf/seaweedfs的installation guide安装weed-fs,见下图:

        

        执行go get github.com/chrislusf/seaweedfs/go/weed命令后,将下载并编译weed-fs,生成的可执行文件weed放在$GOPATH/bin目录,源码存放在$GOPATH/src目录,由于在国内一些网站被墙,所以可能会出现以下错误:

        root@ubuntu:/tmp/chengxf# go get github.com/chrislusf/seaweedfs/go/weed
        package github.com/chrislusf/seaweedfs/go/weed
        imports bazil.org/fuse
        imports bazil.org/fuse/fs
        imports golang.org/x/net/context: unrecognized import path "golang.org/x/net/context"
        # cd .;git clone https://code.google.com/p/go-decimal-inf.exp /tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf
        Cloning into '/tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf'...
        error: Failed to connect to 2404:6800:400a:805::200e: Network is unreachable while accessing https://code.google.com/p/go-decimal-inf.exp/info/refs
        fatal: HTTP request failed
        package github.com/chrislusf/seaweedfs/go/weed
        imports bazil.org/fuse
        imports bazil.org/fuse/fs
        imports github.com/boltdb/bolt
        imports github.com/chrislusf/raft
        imports github.com/gogo/protobuf/proto
        imports github.com/dgrijalva/jwt-go
        imports github.com/gocql/gocql
        imports github.com/golang/groupcache/lru
        imports github.com/golang/snappy/snappy
        imports speter.net/go/exp/math/dec/inf: exit status 128
        package github.com/chrislusf/seaweedfs/go/weed
        imports bazil.org/fuse
        imports bazil.org/fuse/fs
        imports github.com/boltdb/bolt
        imports github.com/chrislusf/raft
        imports github.com/gogo/protobuf/proto
        imports github.com/dgrijalva/jwt-go
        imports github.com/gocql/gocql
        imports github.com/golang/groupcache/lru
        imports github.com/golang/snappy/snappy
        imports github.com/golang/protobuf/proto
        imports github.com/syndtr/goleveldb/leveldb
        imports github.com/syndtr/goleveldb/leveldb/util
        imports github.com/syndtr/gosnappy/snappy
        imports gopkg.in/redis.v2
        imports gopkg.in/bufio.v1
        imports github.com/disintegration/imaging
        imports golang.org/x/image/bmp: unrecognized import path "golang.org/x/image/bmp"
        package github.com/chrislusf/seaweedfs/go/weed
        imports bazil.org/fuse
        imports bazil.org/fuse/fs
        imports github.com/boltdb/bolt
        imports github.com/chrislusf/raft
        imports github.com/gogo/protobuf/proto
        imports github.com/dgrijalva/jwt-go
        imports github.com/gocql/gocql
        imports github.com/golang/groupcache/lru
        imports github.com/golang/snappy/snappy
        imports github.com/golang/protobuf/proto
        imports github.com/syndtr/goleveldb/leveldb
        imports github.com/syndtr/goleveldb/leveldb/util
        imports github.com/syndtr/gosnappy/snappy
        imports gopkg.in/redis.v2
        imports gopkg.in/bufio.v1
        imports github.com/disintegration/imaging
        imports golang.org/x/image/tiff: unrecognized import path "golang.org/x/image/tiff"
        root@ubuntu:/tmp/chengxf# 


        首先上面的unrecognized import path “imports golang.org/x/xxx/xxx”,可以手动去https://github.com/golang上下载相应的包,放置与/tmp/chengxf/gopath/src的相应子目录下,再次执行后,发现只剩如下错误:

        root@ubuntu:/tmp/chengxf# go get github.com/chrislusf/seaweedfs/go/weed
        # cd .;
git clone https://code.google.com/p/go-decimal-inf.exp /tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf
        Cloning into '/tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf'...
        error: Failed to connect to 2404:6800:4005:80b::200e: Network is unreachable while accessing https://code.google.com/p/go-decimal-inf.exp/info/refs
        fatal: HTTP request failed
        package speter.net/go/exp/math/dec/inf: exit status 128
        root@ubuntu:/tmp/chengxf#


        上面是获取code.google.com上的资源出现错误,可以手动去https://github.com/go-inf/inf/tree/master下载源码包,并解压至/tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf目录下,再次执行go get github.com/chrislusf/seaweedfs/go/weed又报错:

        root@ubuntu:/tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf# go get github.com/chrislusf/seaweedfs/go/weed
../../../../../../github.com/gocql/gocql/helpers.go:13:2: code in directory /tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf expects import "gopkg.in/inf.v0"


        好吧,我也不知道什么原因,总之我将/tmp/chengxf/gopath/src/speter.net/go/exp/math/dec/inf/dec.go文件中的”// import "gopkg.in/inf.v0"“删除掉,然后再用go get github.com/chrislusf/seaweedfs/go/weed重新编译就好了。/tmp/chengxf/gopath/bin目录下生成了可执行文件weed。


3. 使用weed-fs存储和访问文件

        准备:在/tmp/chengxf目录下新建如下文件夹:mdata、vol1、vol2、input和output,并在input文件夹中新建测试文件a.txt和b.xml。

        启动master:weed master -mdir="/tmp/chengxf/mdata" -defaultReplication="001" -ip="10.124.69.235" -port=9334 &

        启动volume:weed volume -dir=/tmp/chengxf/vol1/ -mserver="10.124.69.235:9334" -ip="117.78.1.100" -ip.bind="10.124.69.235" -port=8081 -max=5 &

                               weed volume -dir=/tmp/chengxf/vol2/ -mserver="10.124.69.235:9334" -ip="117.78.1.100" -ip.bind="10.124.69.235" -port=8082 -max=5 &

        存储文件:weed upload -server="117.78.1.100:9334" -dir="/tmp/chengxf/input" -include=*.txt

        执行后返回文件fid:[{"fileName":"/tmp/chengxf/input/a.txt","fileUrl":"117.78.1.100:8081/1,01eb8f1e42","fid":"1,01eb8f1e42","size":44}]

        根据fid访问文件:weed download -server="117.78.1.100:9334" -dir="/tmp/chengxf/output" 1,01eb8f1e42

        执行后在/tmp/chengxf/output目录下发现a.txt文件

        测试机器是云服务器,117.78.1.100是公网IP,通过防火墙映射到内网机器10.124.69.235上的,不能直接绑定公网地址,所以采用ip.bind绑定内网地址来实现。

        


有疑问加站长微信联系(非本文作者)

本文来自:CSDN博客

感谢作者:qq286210662

查看原文:golang开源分布式文件系统weed-fs的安装与运用

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:701969077

10443 次点击  
加入收藏 微博
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传