Go常用库-网络框架Gin(1) - 原生http包

一. 写在前面:

• Go提供了原生的Http实现-net/http包,封装的挺好,作为服务器使用把底层的TCP连接,请求报文解析,Header解析,Cookie解析,返回报文组装等功能都实现好了.使用起来只需要写真正逻辑相关的Handler即可.作为客户端使用也提供了默认的Client,只需要构造出请求,即可发送出去,并获得返回.
• 但是,由于定义和实现是绑定在一起的,如果不想用原生的东西,扩展起来可就麻烦了.不花时间啃几千行的server.go和transport.go肯定改不动的.相比之下,感觉javax.servlet就漂亮的多,只是定义好接口,具体怎么实现容器或客户端自行决定;也很容易升级Http版本的支持.

二. net/http包源码简单阅读分析

注: 由于目标只是了解处理流程,对于文件传输(chunk)和HTTP/2的部分,都直接略过了.

1. http子包与关联包

2. http公用源码

Request结构体--(去了注释)

type Request struct {
    Method string
    URL *url.URL
    Proto      string // "HTTP/1.0"
    ProtoMajor int    // 1
    ProtoMinor int    // 0
    Header Header
    Body io.ReadCloser
    GetBody func() (io.ReadCloser, error)
    ContentLength int64
    TransferEncoding []string
    Close bool
    Host string
    Form url.Values
    PostForm url.Values
    MultipartForm *multipart.Form
    Trailer Header
    RemoteAddr string
    TLS *tls.ConnectionState
    Response *Response
    ctx context.Context
}

包含了解析好的URL,eader,Form值;其中Repsonse应该是空的,只有重定向交易才有值.
ctx是后面加入了,为了能优雅进行超时关闭.

Response结构体--(去了注释)

type Response struct {
    Status     string // e.g. "200 OK"
    StatusCode int    // e.g. 200
    Proto      string // e.g. "HTTP/1.0"
    ProtoMajor int    // e.g. 1
    ProtoMinor int    // e.g. 0
    Header Header
    Body io.ReadCloser
    ContentLength int64
    TransferEncoding []string
    Close bool
    Uncompressed bool
    Trailer Header
    // Request is the request that was sent to obtain this Response.
    // Request's Body is nil (having already been consumed).
    // This is only populated for Client requests.
    Request *Request
    TLS *tls.ConnectionState
}

代表了准备返回给Client的Response,其中Body是个ReadCloser

3. http客户端相关源码

transport的核心方法roundTrip(): -- 太长了,略..

client结构为:

type Client struct {
    Transport RoundTripper
    Jar CookieJar
    Timeout time.Duration
}

client东西都藏在Transport里面了,额外增加一个cookie控制和超时.
client的核心方法do(): (这点起名风格很spring啊)

func (c *Client) do(req *Request) (retres *Response, reterr error) {
    if testHookClientDoResult != nil {
        defer func() { testHookClientDoResult(retres, reterr) }()
    }
    if req.URL == nil {
        req.closeBody()
        return nil, &url.Error{
            Op:  urlErrorOp(req.Method),
            Err: errors.New("http: nil Request.URL"),
        }
    }
    // 做了基础的校验后,创建函数变量
    var (
        deadline      = c.deadline()
        reqs          []*Request
        resp          *Response
        copyHeaders   = c.makeHeadersCopier(req)
        reqBodyClosed = false // have we closed the current req.Body?

        // Redirect behavior:
        redirectMethod string
        includeBody    bool
    )
    // 定义错误处理函数
    uerr := func(err error) error {
        // the body may have been closed already by c.send()
        if !reqBodyClosed {
            req.closeBody()
        }
        var urlStr string
        if resp != nil && resp.Request != nil {
            urlStr = stripPassword(resp.Request.URL)
        } else {
            urlStr = stripPassword(req.URL)
        }
        return &url.Error{
            Op:  urlErrorOp(reqs[0].Method),
            URL: urlStr,
            Err: err,
        }
    }
    // 这里直接死循环...
    for {
        // For all but the first request, create the next
        // request hop and replace req.
        // 第一次进来这边是空的,不会进去,知道下面加入reqs才第一次外呼
        if len(reqs) > 0 {
            loc := resp.Header.Get("Location")
            if loc == "" {
                resp.closeBody()
                return nil, uerr(fmt.Errorf("%d response missing Location header", resp.StatusCode))
            }
            u, err := req.URL.Parse(loc)
            if err != nil {
                resp.closeBody()
                return nil, uerr(fmt.Errorf("failed to parse Location header %q: %v", loc, err))
            }
            host := ""
            if req.Host != "" && req.Host != req.URL.Host {
                // If the caller specified a custom Host header and the
                // redirect location is relative, preserve the Host header
                // through the redirect. See issue #22233.
                if u, _ := url.Parse(loc); u != nil && !u.IsAbs() {
                    host = req.Host
                }
            }
            ireq := reqs[0]
            // 重新构造本次轮询请求的Request
            req = &Request{
                Method:   redirectMethod,
                Response: resp,
                URL:      u,
                Header:   make(Header),
                Host:     host,
                Cancel:   ireq.Cancel,
                ctx:      ireq.ctx,
            }
            if includeBody && ireq.GetBody != nil {
                req.Body, err = ireq.GetBody()
                if err != nil {
                    resp.closeBody()
                    return nil, uerr(err)
                }
                req.ContentLength = ireq.ContentLength
            }

            // Copy original headers before setting the Referer,
            // in case the user set Referer on their first request.
            // If they really want to override, they can do it in
            // their CheckRedirect func.
            copyHeaders(req)

            // Add the Referer header from the most recent
            // request URL to the new one, if it's not https->http:
            if ref := refererForURL(reqs[len(reqs)-1].URL, req.URL); ref != "" {
                req.Header.Set("Referer", ref)
            }
            err = c.checkRedirect(req, reqs)

            // Sentinel error to let users select the
            // previous response, without closing its
            // body. See Issue 10069.
            if err == ErrUseLastResponse {
                return resp, nil
            }

            // Close the previous response's body. But
            // read at least some of the body so if it's
            // small the underlying TCP connection will be
            // re-used. No need to check for errors: if it
            // fails, the Transport won't reuse it anyway.
            const maxBodySlurpSize = 2 << 10
            if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
                io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
            }
            resp.Body.Close()

            if err != nil {
                // Special case for Go 1 compatibility: return both the response
                // and an error if the CheckRedirect function failed.
                // See https://golang.org/issue/3795
                // The resp.Body has already been closed.
                ue := uerr(err)
                ue.(*url.Error).URL = loc
                return resp, ue
            }
        }
        // 先把调用存到列表
        reqs = append(reqs, req)
        var err error
        var didTimeout func() bool
        // 这里调用send--用到transport发送报文
        if resp, didTimeout, err = c.send(req, deadline); err != nil {
            // c.send() always closes req.Body
            reqBodyClosed = true
            if !deadline.IsZero() && didTimeout() {
                err = &httpError{
                    // TODO: early in cycle: s/Client.Timeout exceeded/timeout or context cancellation/
                    err:     err.Error() + " (Client.Timeout exceeded while awaiting headers)",
                    timeout: true,
                }
            }
            return nil, uerr(err)
        }

        var shouldRedirect bool
        redirectMethod, shouldRedirect, includeBody = redirectBehavior(req.Method, resp, reqs[0])
        // 这里判断本次发送结果,如果不是重定向,就可以返回了,退出for循环.
        if !shouldRedirect {
            return resp, nil
        }
        // 本次body已经消费掉了,这里要关闭才行.
        req.closeBody()
    }
}

4. http服务端相关源码

Handler的声明

// Handler的接口,只需要实现ServeHTTP即可
type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}
// 定义了HandlerFunc类型,它本身是方法,同时实现了ServeHTTP接口-->实际是调用自身
type HandlerFunc func(ResponseWriter, *Request)

// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
    f(w, r)
}

Server的定义

// A Server defines parameters for running an HTTP server.
// The zero value for Server is a valid configuration.
type Server struct {
    Addr    string  // TCP address to listen on, ":http" if empty
    // Server也只有一个默认的Handler,http.DefaultServeMux
    Handler Handler // handler to invoke, http.DefaultServeMux if nil
    TLSConfig *tls.Config
    ReadTimeout time.Duration
    ReadHeaderTimeout time.Duration
    WriteTimeout time.Duration
    
    IdleTimeout time.Duration
    MaxHeaderBytes int
    TLSNextProto map[string]func(*Server, *tls.Conn, Handler)
    ConnState func(net.Conn, ConnState)
    ErrorLog *log.Logger
    BaseContext func(net.Listener) context.Context
    ConnContext func(ctx context.Context, c net.Conn) context.Context
    
    disableKeepAlives int32     // accessed atomically.
    inShutdown        int32     // accessed atomically (non-zero means we're in Shutdown)
    nextProtoOnce     sync.Once // guards setupHTTP2_* init
    nextProtoErr      error     // result of http2.ConfigureServer if used

    mu         sync.Mutex
    listeners  map[*net.Listener]struct{}
    activeConn map[*conn]struct{}
    doneChan   chan struct{}
    onShutdown []func()
}

从代码层面跟踪一下如何进行服务的处理:

1. 实际上Server并不直接处理Request,从其Serve()方法最后可以看出,它只负责连接的建立

        .......
        tempDelay = 0   
        c := srv.newConn(rw)
        c.setState(c.rwc, StateNew) // before Serve can return
        // *** 注意,这里使用goroutine,也就是说对于每个连接,都是单独的协程独立工作 ***
        go c.serve(ctx)

2. 通过newConn构造出新的conn连接,调用连接的serve方法处理交互,一个conn是可以有多次交互的.
   首先,从连接读取客户端request

for {
        // 在连接还存活的状态下,会一直读取客户端的请求.
        w, err := c.readRequest(ctx)
        ...
            // 解析好Request后,创建serverHandler对象,调用他的ServeHTTP处理请求
            serverHandler{c.server}.ServeHTTP(w, w.req)
    }

然后,调用server的ServeHTTP()方法,找处理的ServeMux

// serverHandler delegates to either the server's Handler or
// DefaultServeMux and also handles "OPTIONS *" requests.
type serverHandler struct {
    srv *Server
}

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
    handler := sh.srv.Handler
    // 默认用Default
    if handler == nil {
        handler = DefaultServeMux
    }
    // 对"*"和"OPTIONS请求特殊处理
    if req.RequestURI == "*" && req.Method == "OPTIONS" {
        handler = globalOptionsHandler{}
    }
    // 调用ServeMux的ServeHTTP方法
    handler.ServeHTTP(rw, req)
}

3. 调用ServeMux的ServeHTTP,找到程序员注册到ServeMux结构中的处理函数

type ServeMux struct {
    mu    sync.RWMutex
    m     map[string]muxEntry
    es    []muxEntry // slice of entries sorted from longest to shortest.
    hosts bool       // whether any patterns contain hostnames
}

type muxEntry struct {
    h       Handler // 这个handler就是我们自定义的处理函数
    pattern string  // 这个pattern就是我们自定义的处理路径
}

可以看出,ServeMux是一个map存储器,把具体的muxEntry保存住

// ServeHTTP dispatches the request to the handler whose
// pattern most closely matches the request URL.
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
    if r.RequestURI == "*" {
        if r.ProtoAtLeast(1, 1) {
            w.Header().Set("Connection", "close")
        }
        w.WriteHeader(StatusBadRequest)
        return
    }
    // 这个方法其实是默认的路由,根据request找具体处理的handler,内部调用了match()方法,实际是一致循环所有muxEntry,然后用strings.HasPrefix判断
    // httprouter说这种方式太慢,用基数树存储结构进行加速.--原生的方法确实不高效
    h, _ := mux.Handler(r)
    // 这里h就是我们调用http.HandleFunc("/hello", hello)时候注册的hello了.
    h.ServeHTTP(w, r)
}

从server代码可以看出,MuxEntry相当于Servlet,负责具体处理,ServerMux是所有Entry的集合,通过match进行router匹配,指派任务.系统提供了默认的Mux统一管理,调用http.handle()实际是想ServerMux进行注册...
ServeHTTP()才是真正具体工作的地方,调用流程为serverHandle-> DefaultServeMux -> 自定义HandlerFunc或者Handler (都叫ServeHTTP,就是为了让人看不懂吗....)

三. httpServer运行分析

好像已经包含在上面的源码分析了...略

四. 文档参考

https://juejin.im/post/5dd11baff265da0c0c1fe813

https://colobu.com/2018/07/25/exposing-go-on-the-internet/

https://cizixs.com/2016/08/17/golang-http-server-side/

有疑问加站长微信联系（非本文作者）