goreplay(gor) golang 流量拷贝工具试用

rongfengliang · · 1755 次点击 · · 开始浏览    
这是一个创建于 的文章,其中的信息可能已经有所发展或是发生改变。

1. 项目地址
https://github.com/buger/goreplay
2. 安装
wget https://github.com/buger/goreplay/releases/download/v0.16.1/gor_0.16.1_x64.tar.gz
tar xvf  gor_0.16.1_x64.tar.gz
cp goreplay /usr/bin
3. 简单使用
//  测试环境使用的是nginx 有连个机器一台进行反向代理,具体的安装配置比较简单

a. 进行流量拷贝并输出到标准输出
goreplay  --input-raw :80 --output-stdout
response message
GET / HTTP/1.1
Host: XXXXXXXXX
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.75 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cache-Control: max-age=0
If-Modified-Since: Sat, 04 Nov 2017 13:52:14 GMT
If-None-Match: W/"59fdc60e-232"
Proxy-Connection: keep-alive
Upgrade-Insecure-Requests: 1
X-Forwarded-For: XXXXXXXXX
X-Lantern-Version: 4.4.0

b. 流量拷贝其他环境

     goreplay --input-raw :80 --output-http="http://xxxxx:port"
     
c. 流量拷贝存储为本地文件(也可以使用s3存储,开源的有minio)
    
     copy:
     goreplay --input-raw :80 --output-file=requests.gor
     replay:
     goreplay --input-file requests.gor --output-http="http://XXXXXXXX:port" 
4. 可用的插件
a. input 

--input-raw - used to capture HTTP traffic, you should specify IP address or 
interface and application port. More about Capturing and replaying traffic.
--input-file - accepts file which previously was recorded using --output-file. 
More about Saving and Replaying from file
--input-tcp - used by Gor aggregation instance if you decided forward traffic 
from multiple forwarder Gor instances to it. 

b. output

--output-http - replay HTTP traffic to given endpoint, accepts base url
--output-file - records incoming traffic to the file. More about Saving and Replaying from file
--output-tcp - forward incoming data to another Gor instance, used in conjunction with --input-tcp. Read more about Aggregator-forwarder setup.
--output-stdout - used for debugging, outputs all data to stdout.
5. 扩展
Rate limiting

a. Limiting replay using absolute number

# staging.server will not get more than ten requests per second
gor --input-tcp :28020 --output-http "http://staging.com|10"

b. Limiting listener using percentage based limiter

# replay server will not get more than 10% of requests 
# useful for high-load environments
gor --input-raw :80 --output-tcp "replay.local:28020|10%"

c. Consistent limiting based on Header or URL param value

# Limit based on header value
gor --input-raw :80 --output-tcp "replay.local:28020|10%" --http-header-limiter "X-API-KEY: 10%"

# Limit based on header value
gor --input-raw :80 --output-tcp "replay.local:28020|10%" --http-param-limiter "api_key: 10%"


Request filtering

a. Allow url regexp

# only forward requests being sent to the /api endpoint
gor --input-raw :8080 --output-http staging.com --http-allow-url /api

b. Disallow url regexp

# only forward requests NOT being sent to the /api... endpoint
gor --input-raw :8080 --output-http staging.com --http-disallow-url /api

c. Filter based on regexp of header

# only forward requests with an api version of 1.0x
gor --input-raw :8080 --output-http staging.com --http-allow-header api-version:^1\.0\d

# only forward requests NOT containing User-Agent header value "Replayed by Gor"
gor --input-raw :8080 --output-http staging.com --http-disallow-header "User-Agent: Replayed by Gor"

d. Filter based on HTTP method

gor --input-raw :80 --output-http "http://staging.server" \
    --http-allow-method GET \
    --http-allow-method OPTIONS
6. 参考资料
// wiki 
https://github.com/buger/goreplay/wiki
https://github.com/buger/goreplay
// kafka
https://github.com/buger/goreplay/wiki/Streaming-from-and-to-Apache-Kafka
// Distributed-configuration
https://github.com/buger/goreplay/wiki/Distributed-configuration

有疑问加站长微信联系(非本文作者)

本文来自:博客园

感谢作者:rongfengliang

查看原文:goreplay(gor) golang 流量拷贝工具试用

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:692541889

1755 次点击  
加入收藏 微博
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传