Go实战--golang中使用HTTPS以及TSL(.crt、.key、.pem区别以及crypto/tls包介绍)

生命不止，继续go go go!!!

HTTP与HTTPS

在WWDC 2016上，苹果在发布iOS 9的同时也向开发者传递了一个消息，那就是到2017年1月1日时App Store中所有应用都必须启用 App Transport Security应用程序安全传输协议，从而提升应用和系统安全性。

HTTPS是Hyper Text Transfer Protocol Secure的缩写，相比http，多了一个secure，这一个secure是怎么来的呢？这是由TLS（SSL）提供的。

https和http都属于应用层，基于TCP（以及UDP）协议。但是不同的是：
HTTP 缺省工作在TCP协议80端口
HTTPS缺省工作在TCP协议443端口

通俗一句话：相比http，https对于大部分人来说，意味着比较安全。

TLS

安全传输层协议（TLS）用于在两个通信应用程序之间提供保密性和数据完整性。
The TLS/SSL is a public/private key infrastructure (PKI). For most common cases, each client and server must have a private key.

TLS与SSL的差异

SSLv2 and SSLv3 are completely different (and both are now considered insecure). SSLv3 and TLSv1.0 are very similar, but have a few differences.

You could consider TLSv1.0 as SSLv3.1

• 版本号：TLS记录格式与SSL记录格式相同，但版本号的值不同，TLS的版本1.0使用的版本号为SSLv3.1。

• 报文鉴别码：SSLv3.0和TLS的MAC算法及MAC计算的范围不同。TLS使用了RFC-2104定义的HMAC算法。SSLv3.0使用了相似的算法，两者差别在于SSLv3.0中，填充字节与密钥之间采用的是连接运算，而HMAC算法采用的是异或运算。但是两者的安全程度是相同的。

• 伪随机函数：TLS使用了称为PRF的伪随机函数来将密钥扩展成数据块，是更安全的方式。

• 报警代码：TLS支持几乎所有的SSLv3.0报警代码，而且TLS还补充定义了很多报警代码，如解密失败（decryption_failed）、记录溢出（record_overflow）、未知CA（unknown_ca）、拒绝访问（access_denied）等。

• 密文族和客户证书：SSLv3.0和TLS存在少量差别，即TLS不支持Fortezza密钥交换、加密算法和客户证书。

• certificate_verify和finished消息：SSLv3.0和TLS在用certificate_verify和finished消息计算MD5和SHA-1散列码时，计算的输入有少许差别，但安全性相当。

• 加密计算：TLS与SSLv3.0在计算主密值（master secret）时采用的方式不同。

• 填充：用户数据加密之前需要增加的填充字节。在SSL中，填充后的数据长度要达到密文块长度的最小整数倍。而在TLS中，填充后的数据长度可以是密文块长度的任意整数倍（但填充的最大长度为255字节），这种方式可以防止基于对报文长度进行分析的攻击。

openssl
openssl(www.openssl.org) 是sslv2,sslv3,tlsv1的一份完整实现,内部包含了大量加密算法程序.其命令行提供了丰富的加密,验证,证书生成等功能,甚至可以用其建立 一个完整的CA.与其同时,它也提供了一套完整的库函数,可用开发用SSL/TLS的通信程序.

插曲：
2016年10月18日，锤子科技CEO罗永浩在锤子手机发布会上，宣布将200多万元门票收入，以及原计划成立的 Smartisan 公益基金近100万元，全部捐赠给 OpenSSL 基金会和 OpenBSD 基金会。

crt、key以及pem的区别以及生成

crt — Alternate synonymous most common among *nix systems .pem (pubkey).

csr — Certficate Signing Requests (synonymous most common among *nix systems).

cer — Microsoft alternate form of .crt, you can use MS to convert .crt to .cer (DER encoded .cer, or base64[PEM] encoded cer).

pem = The PEM extension is used for different types of X.509v3 files which contain ASCII (Base64) armored data prefixed with a «—– BEGIN …» line. These files may also bear the cer or the crt extension.

der — The DER extension is used for binary DER encoded certificates.

证书(Certificate) .cer .crt
私钥(Private Key).key
证书签名请求(Certificate sign request) .csr
至于pem和der，是编码方式，以上三类均可以使用这两种编码方式，因此.pem和.der(少见)不一定是以上三种(Cert,Key,CSR)中的某一种

PEM - Privacy Enhanced Mail,打开看文本格式,以”—–BEGIN…”开头, “—–END…”结尾,内容是BASE64编码.
查看PEM格式证书的信息:openssl x509 -in certificate.pem -text -noout
Apache和*NIX服务器偏向于使用这种编码格式.

DER - Distinguished Encoding Rules,打开看是二进制格式,不可读.
查看DER格式证书的信息:openssl x509 -in certificate.der -inform der -text -noout
Java和Windows服务器偏向于使用这种编码格式.

x509
X.509是一种非常通用的证书格式。所有的证书都符合ITU-T X.509国际标准，因此(理论上)为一种应用创建的证书可以用于任何其他符合X.509标准的应用。

x509证书一般会用到三类文，key，csr，crt。
Key 是私用密钥openssl格，通常是rsa算法。
Csr 是证书请求文件，用于申请证书。在制作csr文件的时，必须使用自己的私钥来签署申，还可以设定一个密钥。
crt是CA认证后的证书文，（windows下面的，其实是crt），签署人用自己的key给你签署的凭证。

生成.key
rsa算法：

openssl genrsa -out server.key 2048

ECDSA算法：

openssl ecparam -genkey -name secp384r1 -out server.key

生成.crt

openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650

需要输入一些信息：

openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:cn
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:wangshubo
Organizational Unit Name (eg, section) []:wangshubo
Common Name (e.g. server FQDN or YOUR name) []:wangshubo
Email Address []:wangshubo1989@126.com

生成pem和key

openssl req -new -nodes -x509 -out server.pem -keyout server.key -days 3650 -subj "/C=DE/ST=NRW/L=Earth/O=Random Company/OU=IT/CN=www.random.com/emailAddress=wangshubo1989@126.com"

crypto/tls包介绍

golang中为我们提供了tls包：
Package tls partially implements TLS 1.2, as specified in RFC 5246.

func LoadX509KeyPair

func LoadX509KeyPair(certFile, keyFile string) (Certificate, error)

LoadX509KeyPair reads and parses a public/private key pair from a pair of files. The files must contain PEM encoded data. The certificate file may contain intermediate certificates following the leaf certificate to form a certificate chain. On successful return, Certificate.Leaf will be nil because the parsed form of the certificate is not retained.

type Config
A Config structure is used to configure a TLS client or server. After one has been passed to a TLS function it must not be modified. A Config may be reused; the tls package will also not modify it.

type Config struct {

        Rand io.Reader

        Time func() time.Time

        Certificates []Certificate

        NameToCertificate map[string]*Certificate

        GetCertificate func(*ClientHelloInfo) (*Certificate, error)

        GetClientCertificate func(*CertificateRequestInfo) (*Certificate, error)

        GetConfigForClient func(*ClientHelloInfo) (*Config, error)

        VerifyPeerCertificate func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error

        RootCAs *x509.CertPool

        NextProtos []string

        ServerName string

        ClientAuth ClientAuthType

        ClientCAs *x509.CertPool

        InsecureSkipVerify bool

        CipherSuites []uint16

        PreferServerCipherSuites bool

        SessionTicketsDisabled bool

        SessionTicketKey [32]byte

        ClientSessionCache ClientSessionCache

        MinVersion uint16

        MaxVersion uint16

        CurvePreferences []CurveID

        DynamicRecordSizingDisabled bool

        Renegotiation RenegotiationSupport

        KeyLogWriter io.Writer
}

这里主要关注一下Certificates，是我们要用到的。

func Listen

func Listen(network, laddr string, config *Config) (net.Listener, error)

Listen creates a TLS listener accepting connections on the given network address using net.Listen. The configuration config must be non-nil and must include at least one certificate or else set GetCertificate.

func Dial

func Dial(network, addr string, config *Config) (*Conn, error)

Dial connects to the given network address using net.Dial and then initiates a TLS handshake, returning the resulting TLS connection. Dial interprets a nil configuration as equivalent to the zero configuration; see the documentation of Config for the defaults.

应用

通过golang生成pem
github上的代码：https://github.com/levigross/go-mutual-tls/blob/master/generate_client_cert.go

// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

// Generate a self-signed X.509 certificate for a TLS server. Outputs to
// 'cert.pem' and 'key.pem' and will overwrite existing files.

package main

import (
    "crypto/ecdsa"
    "crypto/elliptic"
    "crypto/rand"
    "crypto/rsa"
    "crypto/x509"
    "crypto/x509/pkix"
    "encoding/pem"
    "flag"
    "fmt"
    "log"
    "math/big"
    "os"
    "time"
)

var (
    emailAddress = flag.String("email-address", "", "The email address of the user you wish to create the certificate for")
    validFrom    = flag.String("start-date", "", "Creation date formatted as Jan 1 15:04:05 2011")
    validFor     = flag.Duration("duration", 365*24*time.Hour, "Duration that certificate is valid for")
    isCA         = flag.Bool("ca", false, "whether this cert should be its own Certificate Authority")
    rsaBits      = flag.Int("rsa-bits", 2048, "Size of RSA key to generate. Ignored if --ecdsa-curve is set")
    ecdsaCurve   = flag.String("ecdsa-curve", "", "ECDSA curve to use to generate a key. Valid values are P224, P256, P384, P521")
)

func publicKey(priv interface{}) interface{} {
    switch k := priv.(type) {
    case *rsa.PrivateKey:
        return &k.PublicKey
    case *ecdsa.PrivateKey:
        return &k.PublicKey
    default:
        return nil
    }
}

func pemBlockForKey(priv interface{}) *pem.Block {
    switch k := priv.(type) {
    case *rsa.PrivateKey:
        return &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(k)}
    case *ecdsa.PrivateKey:
        b, err := x509.MarshalECPrivateKey(k)
        if err != nil {
            fmt.Fprintf(os.Stderr, "Unable to marshal ECDSA private key: %v", err)
            os.Exit(2)
        }
        return &pem.Block{Type: "EC PRIVATE KEY", Bytes: b}
    default:
        return nil
    }
}

func main() {
    flag.Parse()

    if len(*emailAddress) == 0 {
        log.Fatalf("Missing required --email-address parameter")
    }

    var priv interface{}
    var err error
    switch *ecdsaCurve {
    case "":
        priv, err = rsa.GenerateKey(rand.Reader, *rsaBits)
    case "P224":
        priv, err = ecdsa.GenerateKey(elliptic.P224(), rand.Reader)
    case "P256":
        priv, err = ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
    case "P384":
        priv, err = ecdsa.GenerateKey(elliptic.P384(), rand.Reader)
    case "P521":
        priv, err = ecdsa.GenerateKey(elliptic.P521(), rand.Reader)
    default:
        fmt.Fprintf(os.Stderr, "Unrecognized elliptic curve: %q", *ecdsaCurve)
        os.Exit(1)
    }
    if err != nil {
        log.Fatalf("failed to generate private key: %s", err)
    }

    var notBefore time.Time
    if len(*validFrom) == 0 {
        notBefore = time.Now()
    } else {
        notBefore, err = time.Parse("Jan 2 15:04:05 2006", *validFrom)
        if err != nil {
            fmt.Fprintf(os.Stderr, "Failed to parse creation date: %s\n", err)
            os.Exit(1)
        }
    }

    notAfter := notBefore.Add(*validFor)

    serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
    serialNumber, err := rand.Int(rand.Reader, serialNumberLimit)
    if err != nil {
        log.Fatalf("failed to generate serial number: %s", err)
    }

    template := x509.Certificate{
        SerialNumber: serialNumber,
        Subject: pkix.Name{
            Organization: []string{"Acme Co"},
        },
        NotBefore: notBefore,
        NotAfter:  notAfter,

        KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
        ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth,
            x509.ExtKeyUsageClientAuth},
        BasicConstraintsValid: true,
    }
    template.DNSNames = append(template.DNSNames, "localhost")
    template.EmailAddresses = append(template.EmailAddresses, *emailAddress)

    if *isCA {
        template.IsCA = true
        template.KeyUsage |= x509.KeyUsageCertSign
    }

    derBytes, err := x509.CreateCertificate(rand.Reader, &template, &template, publicKey(priv), priv)
    if err != nil {
        log.Fatalf("Failed to create certificate: %s", err)
    }

    certOut, err := os.Create("cert.pem")
    if err != nil {
        log.Fatalf("failed to open cert.pem for writing: %s", err)
    }
    pem.Encode(certOut, &pem.Block{Type: "CERTIFICATE", Bytes: derBytes})
    certOut.Close()
    log.Print("written cert.pem\n")

    keyOut, err := os.OpenFile("key.pem", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
    if err != nil {
        log.Print("failed to open key.pem for writing:", err)
        return
    }
    pem.Encode(keyOut, pemBlockForKey(priv))
    keyOut.Close()
    log.Print("written key.pem\n")
}

golang中使用HTTPS

package main

import (
    "log"
    "net/http"
)

func HelloServer(w http.ResponseWriter, req *http.Request) {
    w.Header().Set("Content-Type", "text/plain")
    w.Write([]byte("This is an example using https in golang.\n"))
}

func main() {
    http.HandleFunc("/hello", HelloServer)
    err := http.ListenAndServeTLS(":443", "server.crt", "server.key", nil)
    if err != nil {
        log.Fatal("ListenAndServe: ", err)
    }
}

浏览器访问：
https://localhost/hello
不要疑问，出现了访问12306的效果，很正常，因为这是我们自己做的证书。
关于 为何从12306.cn订票时浏览器总是提醒证书不受信任？请看知乎上的讨论，很精彩：
https://www.zhihu.com/question/25334635

golang中使用tls
server.go

package main

import (
    "bufio"
    "crypto/tls"
    "log"
    "net"
)

func main() {
    log.SetFlags(log.Lshortfile)

    cer, err := tls.LoadX509KeyPair("server.crt", "server.key")
    if err != nil {
        log.Println(err)
        return
    }

    config := &tls.Config{Certificates: []tls.Certificate{cer}}
    ln, err := tls.Listen("tcp", ":443", config)
    if err != nil {
        log.Println(err)
        return
    }
    defer ln.Close()

    for {
        conn, err := ln.Accept()
        if err != nil {
            log.Println(err)
            continue
        }
        go handleConnection(conn)
    }
}

func handleConnection(conn net.Conn) {
    defer conn.Close()
    r := bufio.NewReader(conn)
    for {
        msg, err := r.ReadString('\n')
        if err != nil {
            log.Println(err)
            return
        }

        println(msg)

        n, err := conn.Write([]byte("jude\n"))
        if err != nil {
            log.Println(n, err)
            return
        }
    }
}

client.go

package main

import (
    "crypto/tls"
    "log"
)

func main() {
    log.SetFlags(log.Lshortfile)

    conf := &tls.Config{
        InsecureSkipVerify: true,
    }

    conn, err := tls.Dial("tcp", "127.0.0.1:443", conf)
    if err != nil {
        log.Println(err)
        return
    }
    defer conn.Close()

    n, err := conn.Write([]byte("hi\n"))
    if err != nil {
        log.Println(n, err)
        return
    }

    buf := make([]byte, 100)
    n, err = conn.Read(buf)
    if err != nil {
        log.Println(n, err)
        return
    }

    println(string(buf[:n]))
}

这里编写client代码时候需要注意：InsecureSkipVerify: true
也就是说上面的代码中客户端不对服务端的证书进行验证。
go实现的Client端默认也是要对服务端传过来的数字证书进行校验的，但客户端提示：这个证书是由不知名CA签发的！