Golang获取有重定向信息的response的location

reflectsky · · 10411 次点击 · · 开始浏览    
这是一个创建于 的文章,其中的信息可能已经有所发展或是发生改变。

在用golang做模拟登录某个网站的功能时发现了一个问题:如何获取该网站带有重定向信息的Response?或者说我只需要
这个Response中的location信息即可,但是发现对于go的标准库net/http来说如果不hack这个库是没办法直接获取带有重
定向信息的Response,而且目前大多第三方的http库也是基于标准库net/http的,貌似就要改net/http库或库本身就提供了
对于重定向的设置?

那么我们就得先查看一下net/http库的源码了,通常我们都会用http.Client的Do方法发起一个请求,首先看一下它的源码:


// Do sends an HTTP request and returns an HTTP response, following
// policy (e.g. redirects, cookies, auth) as configured on the client.
//
// An error is returned if caused by client policy (such as
// CheckRedirect), or if there was an HTTP protocol error.
// A non-2xx response doesn't cause an error.
//
// When err is nil, resp always contains a non-nil resp.Body.
//
// Callers should close resp.Body when done reading from it. If
// resp.Body is not closed, the Client's underlying RoundTripper
// (typically Transport) may not be able to re-use a persistent TCP
// connection to the server for a subsequent "keep-alive" request.
//
// The request Body, if non-nil, will be closed by the underlying
// Transport, even on errors.
//
// Generally Get, Post, or PostForm will be used instead of Do.
func (c *Client) Do(req *Request) (resp *Response, err error) {
    if req.Method == "GET" || req.Method == "HEAD" {
        return c.doFollowingRedirects(req, shouldRedirectGet)
    }
    if req.Method == "POST" || req.Method == "PUT" {
        return c.doFollowingRedirects(req, shouldRedirectPost)
    }
    return c.send(req)
}

由源码的上方的注释可发现Do方法是不会返回有重定向信息的Response的,只会返回已经重定向跳转成功后的Response,而且会根据是否CheckRedirect返回error云云
那么假设我们要发起的是一个Get请求,所以进入c.doFollowingRedirects()方法中去看看


func (c *Client) doFollowingRedirects(ireq *Request, shouldRedirect func(int) bool) (resp *Response, err error) {
    var base *url.URL
    redirectChecker := c.CheckRedirect
    if redirectChecker == nil {
        redirectChecker = defaultCheckRedirect
    }
    var via []*Request
 
    if ireq.URL == nil {
        ireq.closeBody()
        return nil, errors.New("http: nil Request.URL")
    }
 
    var reqmu sync.Mutex // guards req
    req := ireq
 
    var timer *time.Timer
    if c.Timeout > 0 {
        type canceler interface {
            CancelRequest(*Request)
        }
        tr, ok := c.transport().(canceler)
        if !ok {
            return nil, fmt.Errorf("net/http: Client Transport of type %T doesn't support CancelRequest; Timeout not supported", c.transport())
        }
        timer = time.AfterFunc(c.Timeout, func() {
            reqmu.Lock()
            defer reqmu.Unlock()
            tr.CancelRequest(req)
        })
    }
 
    urlStr := "" // next relative or absolute URL to fetch (after first request)
    redirectFailed := false
    for redirect := 0; ; redirect++ {
        if redirect != 0 {
            nreq := new(Request)
            nreq.Method = ireq.Method
            if ireq.Method == "POST" || ireq.Method == "PUT" {
                nreq.Method = "GET"
            }
            nreq.Header = make(Header)
            nreq.URL, err = base.Parse(urlStr)
            if err != nil {
                break
            }
            if len(via) > 0 {
                // Add the Referer header.
                lastReq := via[len(via)-1]
                if ref := refererForURL(lastReq.URL, nreq.URL); ref != "" {
                    nreq.Header.Set("Referer", ref)
                }
 
                err = redirectChecker(nreq, via)
                if err != nil {
                    redirectFailed = true
                    break
                }
            }
            reqmu.Lock()
            req = nreq
            reqmu.Unlock()
        }
 
        urlStr = req.URL.String()
        if resp, err = c.send(req); err != nil {
            break
        }
 
 
        if shouldRedirect(resp.StatusCode) {
 
            // Read the body if small so underlying TCP connection will be re-used.
            // No need to check for errors: if it fails, Transport won't reuse it anyway.
            const maxBodySlurpSize = 2 << 10
            if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
                io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
            }
            resp.Body.Close()
            if urlStr = resp.Header.Get("Location"); urlStr == "" {
                err = errors.New(fmt.Sprintf("%d response missing Location header", resp.StatusCode))
                break
            }
 
            base = req.URL
            via = append(via, req)
            continue
        }
        if timer != nil {
            resp.Body = &cancelTimerBody{timer, resp.Body}
        }
        return resp, nil
    }
 
    method := ireq.Method
    urlErr := &url .Error{
        Op:  method[0:1] + strings.ToLower(method[1:]),
        URL: urlStr,
        Err: err,
    }
 
    if redirectFailed {
        // Special case for Go 1 compatibility: return both the response
        // and an error if the CheckRedirect function failed.
        // See http://golang.org/issue/3795
        return resp, urlErr
    }
 
    if resp != nil {
        resp.Body.Close()
    }
    return nil, urlErr
}

在上述代码中发现了有检查是否进行重定向的代码:


redirectChecker := c.CheckRedirect
    if redirectChecker == nil {
        redirectChecker = defaultCheckRedirect
    }

redirectChecker是一个这样的函数:func(req Request, via []Request) error
发现了可在client中设置自己的redirectChecker,就是只要实现了func(req Request, via []Request) error即可,其功能是由源码的
defaultCheckRedirect可知是控制重定向跳转的次数而已。


func defaultCheckRedirect(req *Request, via []*Request) error {
    if len(via) >= 10 {
        return errors.New("stopped after 10 redirects")
    }
    return nil
}

defaultCheckRedirect中规定重定向跳转不能超过10次,否则返回error,那么如果我们要禁止跳转重定向的话自己实现一个
CheckRedirect,把其中的10改成0即可。但是即使这样设置后也是不能返回有重定向信息的Response,而是返回一个跳转停
止的error,不过如果你需要的只是带有重定向信息Response中的location信息的话只需要从返回的error中提取即可。http库
client中的doFollowingRedirects中传入了一个shouldRedirect函数,这个函数正是根据各种http协议的代码返回是否进行跳
转的信息,所以如果不改动这部分代码没法直接获取有重定向信息的Response,这样看来http库的设置不够灵活,得自己改或
另外实现了。


有疑问加站长微信联系

本文来自:博客园

感谢作者:reflectsky

查看原文:Golang获取有重定向信息的response的location

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:812540095

10411 次点击  
加入收藏 微博
1 回复  |  直到 2020-03-31 16:12:49
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传