Cleaning up JSON before unmarshaling

polaris · · 230 次点击    
这是一个分享于 的资源,其中的信息可能已经有所发展或是发生改变。
<p>Hi!</p> <p>I&#39;m consuming two separate third-party services. First service uses PascalCase for their keys, and the second uses camelCase. I decided to standardize over snake_case.</p> <p>The payload of one of these services can have up to hundreds of keys.</p> <p>How can I unmarshal camelCase or snake_case, but always marshal snake_case?</p> <hr/>**评论:**<br/><br/>TheMerovius: <pre><p>If you have a field that will always be there, you can define two structs with different fields and use that field to distinguish, which to use: <a href="https://play.golang.org/p/7_YcTY4Qt2" rel="nofollow">https://play.golang.org/p/7_YcTY4Qt2</a></p> <p>You could also unmarshal into a <code>map[string]interface{}</code> and then iterate over that and, depending on the key, manually unmarshal them, or write the equivalent using reflection, or try and find a package that allows you to decode a <code>map[string]interface{}</code> into a struct via reflection (there probably is one).</p> <p>Frankly, this is one of the reasons I&#39;ve always been kind of dissatisfied with the <code>encoding/json</code> API; it treats how to encode/decode as a property of a <em>type</em>, instead of a property of the <em>process</em>. </p></pre>fllr: <pre><p>Yeah. It has been incredibly frustrating. It was disappointing noticing the discrepancies on the apis, for sure, but this should have definitely been easier.</p> <p>Do you have an example of the manual unmarshal? It doesn’t seem at all straight forward is a strongly typed system (I’m coming from python)</p> <p>The two struct solution will unfortunately not work for the api with hundreds of keys</p></pre>TheMerovius: <pre><blockquote> <p>The two struct solution will unfortunately not work for the api with hundreds of keys</p> </blockquote> <p>You are going to have to define a struct anyway. Seems to be a straightforward case for copy-paste, followed by search-and-replace. If you are <em>not</em> defining a struct (i.e. are throwing around maps, which FTR you shouldn&#39;t do), then just use <code>map[string]interface{}</code> and do</p> <pre><code>for k, v := range m { k2 := camel2snake(k) if k != k2 { delete(m, k) m[k] = v } } </code></pre> <p>in your unmarshal function.</p> <blockquote> <p>Do you have an example of the manual unmarshal? It doesn’t seem at all straight forward is a strongly typed system </p> </blockquote> <p>That&#39;s because it isn&#39;t :) If I had a simple example, I would&#39;ve provided it. When I said &#34;manually unmarshal it&#34;, I meant writing the equivalent of</p> <pre><code>var m map[string]interface{} json.Unmarshal(data, &amp;m) foo.SomeKey = m[&#34;some_key&#34;].(string) if v, ok := m[&#34;someKey&#34;]; ok { foo.SomeKey = v.(string) } </code></pre> <p>which, of course, is &#34;not practical for 100s of fields&#34; :)</p> <p>For the reflection-based solution, you want to do essentially the same thing that encoding/json is doing to decide where to store unmarshaled values. Unfortunately, this is non-trivial, which is why I am not-providing an example. If you want to go that route, I&#39;d suggest to start by reading the source code of encoding/json.</p> <p>But I do believe, there is a high likelihood that someone had a similar need and already wrote something like <code>func Map2Value(m map[string]interface{}, v interface{}) error</code>, that you could use like</p> <pre><code>var m map[string]interface{} json.Unmarshal(data, &amp;m) for k, v := range m { k2 := camel2snake(k) if k != k2 { delete(m, k) m[k] = v } } var foo Foo return Map2Value(m, &amp;foo) </code></pre> <p>But as I never had this problem, I don&#39;t know any specific packages and don&#39;t want to recommend any I don&#39;t know. So I&#39;d recommend googling this.</p> <p>Honestly, IMO the &#34;two structs&#34; approach is the most sensible, even if it requires copious amounts of copy-paste. But that&#39;s just… like… um… my opinion. Man. :)</p></pre>fllr: <pre><p>I ended up writing my own UnmarshalJSON function, and it works like a charm! Thanks for the sensible responses! :)</p></pre>drvd: <pre><p>Best advice probably is : Don&#39;t. Keep each API the way it works. A snake_case standard is not helpful at all if dealing with encoding/json.</p></pre>8lall0: <pre><p>I agree. If you can&#39;t guarantee the same api &#34;standard&#34;, it&#39;s just useless work.</p></pre>fllr: <pre><p>What to you mean? It’s not useful to try and consume two different api sets and make something useful out of it?</p></pre>sacrehubert: <pre><blockquote> <p>It’s not useful to try and consume two different api sets and make something useful out of it?</p> </blockquote> <p>That very much depends on what you&#39;re trying to do. We need context in order to give you a good answer.</p></pre>fllr: <pre><p>No. It&#39;s not up to you what is necessary to make my project work.</p></pre>drvd: <pre><blockquote> <p>It’s not useful to try and consume two different api sets and make something useful out of it?</p> </blockquote> <p>It is. But this can be done easier if consumed without further modification or rewriting (even if you call this wasteful rewriting &#34;standardization&#34;).</p></pre>fllr: <pre><p>It&#39;s about keeping a consistent interface inside the project. Consistency matters. And the fact that these two services are differing is important. I found a solution, though.</p></pre>drvd: <pre><p>You consistent interface inside the application does not depend on the details of the external serialization format.</p></pre>fllr: <pre><p>That&#39;s what I&#39;m trying to fix</p></pre>hell_0n_wheel: <pre><blockquote> <p>I decided to standardize </p> </blockquote> <p>You&#39;re committing the classic blunder of &#34;too many standards! I&#39;ll create my own...&#34; <a href="https://xkcd.com/927/" rel="nofollow">as parodied by XKCD</a>.</p> <p>Unless you can give us a very compelling reason for doing so (and so far you haven&#39;t), I&#39;d say this is your mistake right here.</p></pre>fllr: <pre><p>I&#39;m not publishing this standard. This is a standard that will live inside of my environment.</p> <p>Honestly, it should have been to hard to allow my service to consume datasets using multiple different key casings...</p></pre>birkbork: <pre><p>Instead of keeping your internal standard as snake_case keys, you could internally keep a standardized representation of the state using a struct, which can be created from either of your external api:s.</p> <p>This way you can have a &#34;standard&#34; internal structure, that is by no means tied to either of the external api:s</p></pre>fllr: <pre><p>I mean... That is what I&#39;m trying to do. How would I go about doing that?</p></pre>sacrehubert: <pre><p>Why is this necessary?</p></pre>fllr: <pre><p>I have no control over the data coming from the wire, and standardization is important</p></pre>sacrehubert: <pre><blockquote> <p>I have no control over the data coming from the wire</p> </blockquote> <p>Right</p> <blockquote> <p>and standardization is important</p> </blockquote> <p>Why? What happens if it&#39;s not standardized?</p></pre>fllr: <pre><p>I&#39;m going to have to remember whether or not I&#39;m talking to service X or service Y, and thus have to remember if using PascalCase, or camelCase. Either way, it should be simple to make this easy to be consumed however I see it fit. I have 10 years of experience in the industry. I don&#39;t need anyone second guessing the needs I find in my own project just because they don&#39;t know the answer to the question I&#39;m asking.</p></pre>snippet2: <pre><p>You could embed the struct but I&#39;d just keep things simple. Encase you want to add more.</p></pre>

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:701969077

230 次点击  
加入收藏 微博
0 回复
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传