golang https服务简单介绍

首先从启动https监听服务开始.

• 完成Server实例创建.
• 配置https协议
• 启动tcp监听

1. 开启server https服务

通过下边的函数,开启https服务,下边函数主要初始化了Server实例,然后通过ListenAndServeTLS开启https服务.

func ListenAndServeTLS(addr, certFile, keyFile string, handler Handler) error {
    server := &Server{Addr: addr, Handler: handler}
    return server.ListenAndServeTLS(certFile, keyFile)
 }

• Server类,主要是Https服务的参数.下边是这个类的结构体.

type Server struct {
    // TCP address to listen on, ":http" if empty
    // http监听的地址.
    Addr         string    
    // handler to invoke, http.DefaultServeMux if nil
    // 处理客户端http请求的函数
    Handler      Handler    
    // maximum duration before timing out read of the request
    // 读取数据超时时间
    ReadTimeout  time.Duration 
    // maximum duration before timing out write of the response
    // 写入数据超时时间
    WriteTimeout time.Duration 

    // optional TLS config, used by ListenAndServeTLS
    // 安全传输协议配置
    TLSConfig    *tls.Config  

    // MaxHeaderBytes controls the maximum number of bytes the
    // server will read parsing the request header's keys and
    // values, including the request line. It does not limit the
    // size of the request body.
    // If zero, DefaultMaxHeaderBytes is used.
    // 头部最大字节数
    MaxHeaderBytes int

    // TLSNextProto optionally specifies a function to take over
    // ownership of the provided TLS connection when an NPN/ALPN
    // protocol upgrade has occurred. The map key is the protocol
    // name negotiated. The Handler argument should be used to
    // handle HTTP requests and will initialize the Request's TLS
    // and RemoteAddr if not already set. The connection is
    // automatically closed when the function returns.
    // If TLSNextProto is nil, HTTP/2 support is enabled automatically.
    TLSNextProto map[string]func(*Server, *tls.Conn, Handler)

    // ConnState specifies an optional callback function that is
    // called when a client connection changes state. See the
    // ConnState type and associated constants for details.
    ConnState func(net.Conn, ConnState)

    // ErrorLog specifies an optional logger for errors accepting
    // connections and unexpected behavior from handlers.
    // If nil, logging goes to os.Stderr via the log package's
    // standard logger.
    ErrorLog *log.Logger

    disableKeepAlives int32     // accessed atomically.
    nextProtoOnce     sync.Once // guards setupHTTP2_* init
    nextProtoErr      error     // result of http2.ConfigureServer if used
}

1. ListenAndServeTLS过后,开始监听一个端口.循环等待客户端发起tcp请求.主要函数如下:

func (srv *Server) Serve(l net.Listener) error {
    defer l.Close()
    if fn := testHookServerServe; fn != nil {
        fn(srv, l)
    }
    var tempDelay time.Duration // how long to sleep on accept failure

    if err := srv.setupHTTP2_Serve(); err != nil {
        return err
    }

    // TODO: allow changing base context? can't imagine concrete
    // use cases yet.
    // 开启监听的时候,初始化了顶层context
    // 此处是一个空的context,这个context不能被取消.
    baseCtx := context.Background()

    // 创建一个新的context,并设置value值.
    ctx := context.WithValue(baseCtx, ServerContextKey, srv)
    ctx = context.WithValue(ctx, LocalAddrContextKey, l.Addr())
    for {
        rw, e := l.Accept()
        if e != nil {
            if ne, ok := e.(net.Error); ok && ne.Temporary() {
                if tempDelay == 0 {
                    tempDelay = 5 * time.Millisecond
                } else {
                    tempDelay *= 2
                }
                if max := 1 * time.Second; tempDelay > max {
                    tempDelay = max
                }
                srv.logf("http: Accept error: %v; retrying in %v", e, tempDelay)
                time.Sleep(tempDelay)
                continue
            }
            return e
        }
        tempDelay = 0
        c := srv.newConn(rw)
        c.setState(c.rwc, StateNew) // before Serve can return
        go c.serve(ctx)
    }
}

上边过程完成了https服务监听程序. 接下来就可以接收客户端发送过来的请求.

2. server接收client请求

客户端发起连接后,创建出一个TCPConn的实例,这个实例组合了conn,而conn就是每一个客户端创建连接时,产生的一个连接成功的对象.通过conn实例,来与客户端进行报文交互. (下边这个类定义在net/http/server.go中)

// A conn represents the server side of an HTTP connection.
type conn struct {
    // server is the server on which the connection arrived.
    // Immutable; never nil.
    server *Server

    // rwc is the underlying network connection.
    // This is never wrapped by other types and is the value given out
    // to CloseNotifier callers. It is usually of type *net.TCPConn or
    // *tls.Conn.
    rwc net.Conn

    // remoteAddr is rwc.RemoteAddr().String(). It is not populated synchronously
    // inside the Listener's Accept goroutine, as some implementations block.
    // It is populated immediately inside the (*conn).serve goroutine.
    // This is the value of a Handler's (*Request).RemoteAddr.
    remoteAddr string

    // tlsState is the TLS connection state when using TLS.
    // nil means not TLS.
    tlsState *tls.ConnectionState

    // werr is set to the first write error to rwc.
    // It is set via checkConnErrorWriter{w}, where bufw writes.
    werr error

    // r is bufr's read source. It's a wrapper around rwc that provides
    // io.LimitedReader-style limiting (while reading request headers)
    // and functionality to support CloseNotifier. See *connReader docs.
    r *connReader

    // bufr reads from r.
    // Users of bufr must hold mu.
    bufr *bufio.Reader

    // bufw writes to checkConnErrorWriter{c}, which populates werr on error.
    bufw *bufio.Writer

    // lastMethod is the method of the most recent request
    // on this connection, if any.
    lastMethod string

    // mu guards hijackedv, use of bufr, (*response).closeNotifyCh.
    mu sync.Mutex

    // hijackedv is whether this connection has been hijacked
    // by a Handler with the Hijacker interface.
    // It is guarded by mu.
    hijackedv bool
}

• conn类中,引用了net.Conn,他是一个接口,由如下几种方法,net.Conn才是真正与客户端交互的接口.(下边的接口定义在net/net.go包中)

type Conn interface {
    // Read reads data from the connection.
    // Read can be made to time out and return a Error with Timeout() == true
    // after a fixed time limit; see SetDeadline and SetReadDeadline.
    Read(b []byte) (n int, err error)

    // Write writes data to the connection.
    // Write can be made to time out and return a Error with Timeout() == true
    // after a fixed time limit; see SetDeadline and SetWriteDeadline.
    Write(b []byte) (n int, err error)

    // Close closes the connection.
    // Any blocked Read or Write operations will be unblocked and return errors.
    Close() error

    // LocalAddr returns the local network address.
    LocalAddr() Addr

    // RemoteAddr returns the remote network address.
    RemoteAddr() Addr

    // SetDeadline sets the read and write deadlines associated
    // with the connection. It is equivalent to calling both
    // SetReadDeadline and SetWriteDeadline.
    //
    // A deadline is an absolute time after which I/O operations
    // fail with a timeout (see type Error) instead of
    // blocking. The deadline applies to all future I/O, not just
    // the immediately following call to Read or Write.
    //
    // An idle timeout can be implemented by repeatedly extending
    // the deadline after successful Read or Write calls.
    //
    // A zero value for t means I/O operations will not time out.
    SetDeadline(t time.Time) error

    // SetReadDeadline sets the deadline for future Read calls.
    // A zero value for t means Read will not time out.
    SetReadDeadline(t time.Time) error

    // SetWriteDeadline sets the deadline for future Write calls.
    // Even if write times out, it may return n > 0, indicating that
    // some of the data was successfully written.
    // A zero value for t means Write will not time out.
    SetWriteDeadline(t time.Time) error
}

• 通过下边的函数来处理客户端的https请求,用户每发起一个请求,都会创建一个goroutine. 这个新创建的goroutine负责与客户端进行交互. net/http/server.go中定义的conn中使用了net/net.go中的接口来与客户端实现报文通信.

func (c *conn) serve(ctx context.Context) {
    // 客户端地址.
    c.remoteAddr = c.rwc.RemoteAddr().String()

    // 异常补货,当一个用户的请求导致服务器异常后,http监听进程不会退出.
    defer func() {
        if err := recover(); err != nil {
            const size = 64 << 10
            buf := make([]byte, size)
            buf = buf[:runtime.Stack(buf, false)]
            c.server.logf("http: panic serving %v: %v\n%s", c.remoteAddr, err, buf)
        }
        if !c.hijacked() {
            c.close()
            c.setState(c.rwc, StateClosed)
        }
    }()

    // tlsConn 就是实现了net/net.go中conn接口的一个实例.
    if tlsConn, ok := c.rwc.(*tls.Conn); ok {
        if d := c.server.ReadTimeout; d != 0 {
            // 设置读超时时间
            c.rwc.SetReadDeadline(time.Now().Add(d))
        }
        if d := c.server.WriteTimeout; d != 0 {
            // 设置写超时时间
            c.rwc.SetWriteDeadline(time.Now().Add(d))
        }
        if err := tlsConn.Handshake(); err != nil {
            c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err)
            return
        }
        c.tlsState = new(tls.ConnectionState)
        *c.tlsState = tlsConn.ConnectionState()
        if proto := c.tlsState.NegotiatedProtocol; validNPN(proto) {
            if fn := c.server.TLSNextProto[proto]; fn != nil {
                h := initNPNRequest{tlsConn, serverHandler{c.server}}
                fn(c.server, tlsConn, h)
            }
            return
        }
    }

    // HTTP/1.x from here on.

    c.r = &connReader{r: c.rwc}
    c.bufr = newBufioReader(c.r)
    c.bufw = newBufioWriterSize(checkConnErrorWriter{c}, 4<<10)

    ctx, cancelCtx := context.WithCancel(ctx)
    defer cancelCtx()

    for {
        w, err := c.readRequest(ctx)
        if c.r.remain != c.server.initialReadLimitSize() {
            // If we read any bytes off the wire, we're active.
            c.setState(c.rwc, StateActive)
        }
        if err != nil {
            if err == errTooLarge {
                // Their HTTP client may or may not be
                // able to read this if we're
                // responding to them and hanging up
                // while they're still writing their
                // request. Undefined behavior.
                io.WriteString(c.rwc, "HTTP/1.1 431 Request Header Fields Too Large\r\nContent-Type: text/plain\r\nConnection: close\r\n\r\n431 Request Header Fields Too Large")
                c.closeWriteAndWait()
                return
            }
            if err == io.EOF {
                return // don't reply
            }
            if neterr, ok := err.(net.Error); ok && neterr.Timeout() {
                return // don't reply
            }
            var publicErr string
            if v, ok := err.(badRequestError); ok {
                publicErr = ": " + string(v)
            }
            io.WriteString(c.rwc, "HTTP/1.1 400 Bad Request\r\nContent-Type: text/plain\r\nConnection: close\r\n\r\n400 Bad Request"+publicErr)
            return
        }

        // Expect 100 Continue support
        req := w.req
        if req.expectsContinue() {
            if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 {
                // Wrap the Body reader with one that replies on the connection
                req.Body = &expectContinueReader{readCloser: req.Body, resp: w}
            }
        } else if req.Header.get("Expect") != "" {
            w.sendExpectationFailed()
            return
        }

        // HTTP cannot have multiple simultaneous active requests.[*]
        // Until the server replies to this request, it can't read another,
        // so we might as well run the handler in this goroutine.
        // [*] Not strictly true: HTTP pipelining. We could let them all process
        // in parallel even if their responses need to be serialized.
        serverHandler{c.server}.ServeHTTP(w, w.req)
        w.cancelCtx()
        if c.hijacked() {
            return
        }
        w.finishRequest()
        if !w.shouldReuseConnection() {
            if w.requestBodyLimitHit || w.closedRequestBodyEarly() {
                c.closeWriteAndWait()
            }
            return
        }
        c.setState(c.rwc, StateIdle)
    }
}

• 通过上边的代码可知request是response中的一部分.第三方路由中,自定义了serverHandler{c.server}.ServeHTTP(w, w.req)部分,从而脱离了SDK默认的路由方式,使用自定义的路由处理规则.
• 用户若要自定义路由处理函数,需要实现Handler接口.接口定义如下.

type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

• 用户可以自定义一个类,实现Handler接口中的ServeHTTP方法.就可以了.
• ServeHTTP方法接收ResponseWrite与Request.其中ResponseWrite是一个接口,Response实现了这个接口.其接口定义如下:

type response struct {
    conn             *conn
    req              *Request // request for this response
    reqBody          io.ReadCloser
    cancelCtx        context.CancelFunc // when ServeHTTP exits
    wroteHeader      bool               // reply header has been (logically) written
    wroteContinue    bool               // 100 Continue response was written
    wants10KeepAlive bool               // HTTP/1.0 w/ Connection "keep-alive"
    wantsClose       bool               // HTTP request has Connection "close"

    w  *bufio.Writer // buffers output in chunks to chunkWriter
    cw chunkWriter

    // handlerHeader is the Header that Handlers get access to,
    // which may be retained and mutated even after WriteHeader.
    // handlerHeader is copied into cw.header at WriteHeader
    // time, and privately mutated thereafter.
    handlerHeader Header
    calledHeader  bool // handler accessed handlerHeader via Header

    written       int64 // number of bytes written in body
    contentLength int64 // explicitly-declared Content-Length; or -1
    status        int   // status code passed to WriteHeader

    // close connection after this reply.  set on request and
    // updated after response from handler if there's a
    // "Connection: keep-alive" response header and a
    // Content-Length.
    closeAfterReply bool

    // requestBodyLimitHit is set by requestTooLarge when
    // maxBytesReader hits its max size. It is checked in
    // WriteHeader, to make sure we don't consume the
    // remaining request body to try to advance to the next HTTP
    // request. Instead, when this is set, we stop reading
    // subsequent requests on this connection and stop reading
    // input from it.
    requestBodyLimitHit bool

    // trailers are the headers to be sent after the handler
    // finishes writing the body. This field is initialized from
    // the Trailer response header when the response header is
    // written.
    trailers []string

    handlerDone atomicBool // set true when the handler exits

    // Buffers for Date and Content-Length
    dateBuf [len(TimeFormat)]byte
    clenBuf [10]byte

    // closeNotifyCh is non-nil once CloseNotify is called.
    // Guarded by conn.mu
    closeNotifyCh <-chan bool
}

Response中可以获取到Request. Request定义如下:

type Request struct {
    // Method specifies the HTTP method (GET, POST, PUT, etc.).
    // For client requests an empty string means GET.
    Method string

    // URL specifies either the URI being requested (for server
    // requests) or the URL to access (for client requests).
    //
    // For server requests the URL is parsed from the URI
    // supplied on the Request-Line as stored in RequestURI.  For
    // most requests, fields other than Path and RawQuery will be
    // empty. (See RFC 2616, Section 5.1.2)
    //
    // For client requests, the URL's Host specifies the server to
    // connect to, while the Request's Host field optionally
    // specifies the Host header value to send in the HTTP
    // request.
    URL *url.URL

    // The protocol version for incoming server requests.
    //
    // For client requests these fields are ignored. The HTTP
    // client code always uses either HTTP/1.1 or HTTP/2.
    // See the docs on Transport for details.
    Proto      string // "HTTP/1.0"
    ProtoMajor int    // 1
    ProtoMinor int    // 0

    // Header contains the request header fields either received
    // by the server or to be sent by the client.
    //
    // If a server received a request with header lines,
    //
    //  Host: example.com
    //  accept-encoding: gzip, deflate
    //  Accept-Language: en-us
    //  fOO: Bar
    //  foo: two
    //
    // then
    //
    //  Header = map[string][]string{
    //      "Accept-Encoding": {"gzip, deflate"},
    //      "Accept-Language": {"en-us"},
    //      "Foo": {"Bar", "two"},
    //  }
    //
    // For incoming requests, the Host header is promoted to the
    // Request.Host field and removed from the Header map.
    //
    // HTTP defines that header names are case-insensitive. The
    // request parser implements this by using CanonicalHeaderKey,
    // making the first character and any characters following a
    // hyphen uppercase and the rest lowercase.
    //
    // For client requests, certain headers such as Content-Length
    // and Connection are automatically written when needed and
    // values in Header may be ignored. See the documentation
    // for the Request.Write method.
    Header Header

    // Body is the request's body.
    //
    // For client requests a nil body means the request has no
    // body, such as a GET request. The HTTP Client's Transport
    // is responsible for calling the Close method.
    //
    // For server requests the Request Body is always non-nil
    // but will return EOF immediately when no body is present.
    // The Server will close the request body. The ServeHTTP
    // Handler does not need to.
    Body io.ReadCloser

    // ContentLength records the length of the associated content.
    // The value -1 indicates that the length is unknown.
    // Values >= 0 indicate that the given number of bytes may
    // be read from Body.
    // For client requests, a value of 0 means unknown if Body is not nil.
    ContentLength int64

    // TransferEncoding lists the transfer encodings from outermost to
    // innermost. An empty list denotes the "identity" encoding.
    // TransferEncoding can usually be ignored; chunked encoding is
    // automatically added and removed as necessary when sending and
    // receiving requests.
    TransferEncoding []string

    // Close indicates whether to close the connection after
    // replying to this request (for servers) or after sending this
    // request and reading its response (for clients).
    //
    // For server requests, the HTTP server handles this automatically
    // and this field is not needed by Handlers.
    //
    // For client requests, setting this field prevents re-use of
    // TCP connections between requests to the same hosts, as if
    // Transport.DisableKeepAlives were set.
    Close bool

    // For server requests Host specifies the host on which the
    // URL is sought. Per RFC 2616, this is either the value of
    // the "Host" header or the host name given in the URL itself.
    // It may be of the form "host:port".
    //
    // For client requests Host optionally overrides the Host
    // header to send. If empty, the Request.Write method uses
    // the value of URL.Host.
    Host string

    // Form contains the parsed form data, including both the URL
    // field's query parameters and the POST or PUT form data.
    // This field is only available after ParseForm is called.
    // The HTTP client ignores Form and uses Body instead.
    Form url.Values

    // PostForm contains the parsed form data from POST, PATCH,
    // or PUT body parameters.
    //
    // This field is only available after ParseForm is called.
    // The HTTP client ignores PostForm and uses Body instead.
    PostForm url.Values

    // MultipartForm is the parsed multipart form, including file uploads.
    // This field is only available after ParseMultipartForm is called.
    // The HTTP client ignores MultipartForm and uses Body instead.
    MultipartForm *multipart.Form

    // Trailer specifies additional headers that are sent after the request
    // body.
    //
    // For server requests the Trailer map initially contains only the
    // trailer keys, with nil values. (The client declares which trailers it
    // will later send.)  While the handler is reading from Body, it must
    // not reference Trailer. After reading from Body returns EOF, Trailer
    // can be read again and will contain non-nil values, if they were sent
    // by the client.
    //
    // For client requests Trailer must be initialized to a map containing
    // the trailer keys to later send. The values may be nil or their final
    // values. The ContentLength must be 0 or -1, to send a chunked request.
    // After the HTTP request is sent the map values can be updated while
    // the request body is read. Once the body returns EOF, the caller must
    // not mutate Trailer.
    //
    // Few HTTP clients, servers, or proxies support HTTP trailers.
    Trailer Header

    // RemoteAddr allows HTTP servers and other software to record
    // the network address that sent the request, usually for
    // logging. This field is not filled in by ReadRequest and
    // has no defined format. The HTTP server in this package
    // sets RemoteAddr to an "IP:port" address before invoking a
    // handler.
    // This field is ignored by the HTTP client.
    RemoteAddr string

    // RequestURI is the unmodified Request-URI of the
    // Request-Line (RFC 2616, Section 5.1) as sent by the client
    // to a server. Usually the URL field should be used instead.
    // It is an error to set this field in an HTTP client request.
    RequestURI string

    // TLS allows HTTP servers and other software to record
    // information about the TLS connection on which the request
    // was received. This field is not filled in by ReadRequest.
    // The HTTP server in this package sets the field for
    // TLS-enabled connections before invoking a handler;
    // otherwise it leaves the field nil.
    // This field is ignored by the HTTP client.
    TLS *tls.ConnectionState

    // Cancel is an optional channel whose closure indicates that the client
    // request should be regarded as canceled. Not all implementations of
    // RoundTripper may support Cancel.
    //
    // For server requests, this field is not applicable.
    //
    // Deprecated: Use the Context and WithContext methods
    // instead. If a Request's Cancel field and context are both
    // set, it is undefined whether Cancel is respected.
    Cancel <-chan struct{}

    // Response is the redirect response which caused this request
    // to be created. This field is only populated during client
    // redirects.
    Response *Response

    // ctx is either the client or server context. It should only
    // be modified via copying the whole Request using WithContext.
    // It is unexported to prevent people from using Context wrong
    // and mutating the contexts held by callers of the same request.
    ctx context.Context
}

• 在Response与Request中可以获取到客户端与服务端的参数信息.所以,在自定义的路由处理函数中,通过这两个变量,可以轻松的完成client与server之间的通信处理.
• 在这个地方插入中间件,可以实现附加的功能处理.

总结:

golang sdk http服务支持自定义的路由处理函数. 这也是很多中间件注入的地方.通过中间件可以完成很多系统需要的附加功能.如权限认证,缓存处理,负载均衡等等