在用golang做模拟登录某个网站的功能时发现了一个问题:如何获取该网站带有重定向信息的Response?或者说我只需要
这个Response中的location信息即可,但是发现对于go的标准库net/http来说如果不hack这个库是没办法直接获取带有重
定向信息的Response,而且目前大多第三方的http库也是基于标准库net/http的,貌似就要改net/http库或库本身就提供了
对于重定向的设置?
那么我们就得先查看一下net/http库的源码了,通常我们都会用http.Client的Do方法发起一个请求,首先看一下它的源码:
// Do sends an HTTP request and returns an HTTP response, following
// policy (e.g. redirects, cookies, auth) as configured on the client.
//
// An error is returned if caused by client policy (such as
// CheckRedirect), or if there was an HTTP protocol error.
// A non-2xx response doesn't cause an error.
//
// When err is nil, resp always contains a non-nil resp.Body.
//
// Callers should close resp.Body when done reading from it. If
// resp.Body is not closed, the Client's underlying RoundTripper
// (typically Transport) may not be able to re-use a persistent TCP
// connection to the server for a subsequent "keep-alive" request.
//
// The request Body, if non-nil, will be closed by the underlying
// Transport, even on errors.
//
// Generally Get, Post, or PostForm will be used instead of Do.
func (c *Client) Do(req *Request) (resp *Response, err error) {
if req.Method == "GET" || req.Method == "HEAD" {
return c.doFollowingRedirects(req, shouldRedirectGet)
}
if req.Method == "POST" || req.Method == "PUT" {
return c.doFollowingRedirects(req, shouldRedirectPost)
}
return c.send(req)
}
由源码的上方的注释可发现Do方法是不会返回有重定向信息的Response的,只会返回已经重定向跳转成功后的Response,而且会根据是否CheckRedirect返回error云云
那么假设我们要发起的是一个Get请求,所以进入c.doFollowingRedirects()方法中去看看
func (c *Client) doFollowingRedirects(ireq *Request, shouldRedirect func(int) bool) (resp *Response, err error) {
var base *url.URL
redirectChecker := c.CheckRedirect
if redirectChecker == nil {
redirectChecker = defaultCheckRedirect
}
var via []*Request
if ireq.URL == nil {
ireq.closeBody()
return nil, errors.New("http: nil Request.URL")
}
var reqmu sync.Mutex // guards req
req := ireq
var timer *time.Timer
if c.Timeout > 0 {
type canceler interface {
CancelRequest(*Request)
}
tr, ok := c.transport().(canceler)
if !ok {
return nil, fmt.Errorf("net/http: Client Transport of type %T doesn't support CancelRequest; Timeout not supported", c.transport())
}
timer = time.AfterFunc(c.Timeout, func() {
reqmu.Lock()
defer reqmu.Unlock()
tr.CancelRequest(req)
})
}
urlStr := "" // next relative or absolute URL to fetch (after first request)
redirectFailed := false
for redirect := 0; ; redirect++ {
if redirect != 0 {
nreq := new(Request)
nreq.Method = ireq.Method
if ireq.Method == "POST" || ireq.Method == "PUT" {
nreq.Method = "GET"
}
nreq.Header = make(Header)
nreq.URL, err = base.Parse(urlStr)
if err != nil {
break
}
if len(via) > 0 {
// Add the Referer header.
lastReq := via[len(via)-1]
if ref := refererForURL(lastReq.URL, nreq.URL); ref != "" {
nreq.Header.Set("Referer", ref)
}
err = redirectChecker(nreq, via)
if err != nil {
redirectFailed = true
break
}
}
reqmu.Lock()
req = nreq
reqmu.Unlock()
}
urlStr = req.URL.String()
if resp, err = c.send(req); err != nil {
break
}
if shouldRedirect(resp.StatusCode) {
// Read the body if small so underlying TCP connection will be re-used.
// No need to check for errors: if it fails, Transport won't reuse it anyway.
const maxBodySlurpSize = 2 << 10
if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
}
resp.Body.Close()
if urlStr = resp.Header.Get("Location"); urlStr == "" {
err = errors.New(fmt.Sprintf("%d response missing Location header", resp.StatusCode))
break
}
base = req.URL
via = append(via, req)
continue
}
if timer != nil {
resp.Body = &cancelTimerBody{timer, resp.Body}
}
return resp, nil
}
method := ireq.Method
urlErr := &url .Error{
Op: method[0:1] + strings.ToLower(method[1:]),
URL: urlStr,
Err: err,
}
if redirectFailed {
// Special case for Go 1 compatibility: return both the response
// and an error if the CheckRedirect function failed.
// See http://golang.org/issue/3795
return resp, urlErr
}
if resp != nil {
resp.Body.Close()
}
return nil, urlErr
}
在上述代码中发现了有检查是否进行重定向的代码:
redirectChecker := c.CheckRedirect
if redirectChecker == nil {
redirectChecker = defaultCheckRedirect
}
redirectChecker是一个这样的函数:func(req Request, via []Request) error
发现了可在client中设置自己的redirectChecker,就是只要实现了func(req Request, via []Request) error即可,其功能是由源码的
defaultCheckRedirect可知是控制重定向跳转的次数而已。
func defaultCheckRedirect(req *Request, via []*Request) error {
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
}
return nil
}
defaultCheckRedirect中规定重定向跳转不能超过10次,否则返回error,那么如果我们要禁止跳转重定向的话自己实现一个
CheckRedirect,把其中的10改成0即可。但是即使这样设置后也是不能返回有重定向信息的Response,而是返回一个跳转停
止的error,不过如果你需要的只是带有重定向信息Response中的location信息的话只需要从返回的error中提取即可。http库
client中的doFollowingRedirects中传入了一个shouldRedirect函数,这个函数正是根据各种http协议的代码返回是否进行跳
转的信息,所以如果不改动这部分代码没法直接获取有重定向信息的Response,这样看来http库的设置不够灵活,得自己改或
另外实现了。