net/http issue setting a cookie with value 本

<p>I get this error when trying to set that character to a value in the cookie</p> <pre><code>net/http: invalid byte &#39;æ&#39; in Cookie.Value; dropping invalid bytes </code></pre> <p>with the code being </p> <pre><code>cookieString := &#34;本&#34; cookie := http.Cookie{Name: &#34;characters&#34;, Value: cookieString} http.SetCookie(w, &amp;cookie) </code></pre> <hr/>**评论：**<br/><br/>tv64738: <pre><p><a href="https://tools.ietf.org/html/rfc6265#section-4.1.1">https://tools.ietf.org/html/rfc6265#section-4.1.1</a></p> <pre><code> set-cookie-header = &#34;Set-Cookie:&#34; SP set-cookie-string set-cookie-string = cookie-pair *( &#34;;&#34; SP cookie-av ) cookie-pair = cookie-name &#34;=&#34; cookie-value cookie-name = token cookie-value = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE ) cookie-octet = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E ; US-ASCII characters excluding CTLs, ; whitespace DQUOTE, comma, semicolon, ; and backslash </code></pre> <p>The usual thing to do is to base64-encode it.</p></pre>irregular_regular: <pre><p>I can kind of see the documentation is basically a regular language. So essentially the cookie value can only be US-ASCII characters which excludes most UTF-8 characters</p></pre>tscs37: <pre><p>Yes. This is a problem of HTTP itself, it&#39;s all US-ASCII only and everything else needs to be encoded in base64 (or something else that only uses ASCII characters).</p> <p>You&#39;ll see similar problems when using SMTP/IMAP (or Email in general) and most plaintext file encodings standardized before this century.</p></pre>iroflmaowtf: <pre><blockquote> <p>So essentially the cookie value can only be US-ASCII characters</p> </blockquote> <p>correct, basically, you want to make sure that whatever goes within the http header is ASCII, anything that might come from user input, config files and/or database, I&#39;d encode to base64 and stay on the safe side</p></pre>jerf: <pre><p>Edit: Ignore this comment, it&#39;s all wrong; leaving only so the thread makes sense.</p> <p>Technically base64 also uses unsafe characters according to the standard, but I think it&#39;ll work in all browsers since so many people do it.</p> <p>If you&#39;re motivated you can use an encoding scheme that is still safe. It&#39;s actually pretty simple to set it up in go: <a href="https://github.com/thejerf/sphyraena/blob/master/secret/secret.go#L25" rel="nofollow">as so</a>. (I don&#39;t recommend anything else from that repo but you can take that as-is.)</p></pre>thornag: <pre><p>Or you use stdlib for this and do</p> <p>base64.URLEncoding.EncodeToString([]byte(value)) that will produce base64 according to <a href="https://en.wikipedia.org/wiki/Base64#URL_applications" rel="nofollow">https://en.wikipedia.org/wiki/Base64#URL_applications</a></p></pre>jerf: <pre><p>For some reason I thought that included illegal characters, but I seem to be incorrect. So your comment does not become unhinged from the context I&#39;ll leave my comment, but this is better.</p> <p>Now I really don&#39;t know what I was thinking, since it seems standard base64 encoding is valid too. Perhaps I had the wrong cookie standard.</p></pre>thornag: <pre><p>I believe standard encoding is not safe as it may contain characters breaking parsing such as / or =.</p></pre>jerf: <pre><pre><code>const encodeStd = &#34;ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/&#34; </code></pre> <p>I thought so too, but + is 0x2b and / is 0x2f, and + is in the second block of the grammar (<code>%x23-2B</code>) and slash in the third (<code>%x2D-3A</code>).</p> <p>According to the comments in the code I made where I wrote this stuff in the first place, I got my ideas from <a href="https://tools.ietf.org/html/rfc2616#section-2.2" rel="nofollow">RFC2616&#39;s token construction</a>, which says if you have separators, which includes a /, you need to quote the value in the HTTP header for it to be legit. But if you quote it, which the net/http library ought to be doing for you, the standard base64 is usable.</p> <p>Well. I&#39;ve learned some stuff today. It seems I&#39;ve got some code to tweak. And in another thread I&#39;m learning all sorts of things about the Context object I&#39;ve been screwing up....</p></pre>irregular_regular: <pre><p>Which thread out of curiosity?</p></pre>jerf: <pre><p><a href="https://www.reddit.com/r/golang/comments/747siz/linked_lists_as_map_keys_a_weird_go_trick/" rel="nofollow">https://www.reddit.com/r/golang/comments/747siz/linked_lists_as_map_keys_a_weird_go_trick/</a></p></pre>