Is this an appropriate situation in which to use Mutexes?

xuanbao · · 206 次点击    
这是一个分享于 的资源,其中的信息可能已经有所发展或是发生改变。
<p>I&#39;m Go noob, and trying to write some functions associated with message handler callbacks for the Paho MQTT client (i.e. what to do when a message is received by the client). This is a two part question, and I&#39;ve tried to clearly indicate both parts:</p> <p>I need to check the payload data against a whitelist <code>map</code> of allowed data before deciding to save to a database or not. I&#39;m trying to accomplish two things by this:</p> <ul> <li>Detect bad data with a few cheap CPU cycles rather than a comparatively expensive database hit.</li> <li>Store the associated prepared <code>INSERT</code> statements in my <code>map</code> so that they <code>Exec</code> faster.</li> </ul> <p><strong>Q1: Is this a sensible way of doing this?</strong> I anticipate having perhaps a few thousand entries in that <code>map</code>, and never call <code>Close()</code> on the statements.</p> <p>I also need to poll the database occasionally to update the <code>map</code> (it&#39;s called &#39;topics&#39;), as what constitutes valid data is subject to change. At the moment I&#39;ve got something like this (removed error checking for simplicity here):</p> <pre><code>topics := make(map[string]*sql.Stmt) tc := time.NewTicker(time.Duration(freq)*time.Second).C // set up a ticker channel to poll every &#39;freq&#39; seconds go func() { for { select { case &lt;- tc: // wipe the topics map and create a new one. TODO: does this need a mutex?! topics = make(map[string]*sql.Stmt) // poll the devices and make their prepared statements in a separate goroutine go func() { var mk string // to store the map keys, referred to as &#39;identifier&#39; in the SQL query rows, _ := db.Query(&#34;SELECT identifier FROM message_publishers WHERE active;&#34;) defer rows.Close() for rows.Next() { _ := rows.Scan(&amp;mk) // in below SQL the table name is derived from the getter getTableName insert_stmt, _ := m.tsdb.Prepare(fmt.Sprintf(&#34;INSERT INTO \&#34;%s\&#34;(colName) VALUES($1);&#34;, getTableName(mk))) topics[mk] = insert_stmt } }() } } }() </code></pre> <p><strong>Q2: Do I need a mutex lock on that <code>map</code>? (on the last line)</strong> I&#39;m going to want to read from it (when a message arrives) while the goroutine is populating it with updated <code>sql.Stmt</code> values. I&#39;m also two goroutines deep; am I doing this right, or should I take another approach?</p> <hr/>**评论:**<br/><br/>kapoof_euw: <pre><p>A few things:</p> <ul> <li>You can use a &#39;for range&#39; statement instead of a for with a select inside. Just &#39;for range tc&#39; if I&#39;m not mistaken.</li> <li>I would be careful with spawning new go-routines on every tick. If your DB access would be slow or you have to process a lot of rows, your go-routine might not be done before the next tick. Having 2 of those work simultaneously might give incorrect results. You can get away with omitting the inner &#34;go func&#34; and just executing it in the same thread as your for-loop.</li> <li>Instead of populating your &#34;active&#34; map, I would suggest you populate a different map stored in a temporary var first and then replacing your &#34;active map&#34; with the one stored in the temporary var. So basically instead of the &#34;wipe the topics map&#34;, just use a different map and replace it when it&#39;s filled up. That way you&#39;ll never have concurrent read-write to the same map and you don&#39;t need a mutex.</li> </ul></pre>yorbit: <pre><p>Great advice; thanks so much for reading the code through.</p></pre>metamatic: <pre><p>A map and a mutex is a perfectly reasonable way to implement some caching. Yes, you need to lock read and well as write. You might fall under the preconditions for which <a href="" rel="nofollow">sync.Map</a> is optimized.</p> <p>Assuming you know you&#39;re going to use all those SQL statements repeatedly for every new publisher, it probably makes sense to prepare and cache the statements. However, I&#39;m thinking you&#39;d get a much bigger boost in performance from buffering and inserting data in batches rather than paying the overhead of journal and index updates for every individual insert. That&#39;s harder work from a programming point of view, of course.</p></pre>yorbit: <pre><p>Thanks for the tip on <code>sync.Map</code>, I didn&#39;t know about that type. Regarding batching, that&#39;s a good idea, but as you pointed out, it&#39;s extra work to buffer for each table. I&#39;ll look into it when I feel it could become a bottleneck.</p></pre>MrPhatBob: <pre><p>I can&#39;t remember where I got this Mutex protected Map code from but if you combine the Mutex with the Map in a struct:</p> <pre><code>var index = struct { sync.RWMutex m map[string]string } {m: make(map[string]string)} </code></pre> <p>Then you can protect writes with:</p> <pre><code>index.Lock() index.m[idx] = thing you&#39;re writing index.Unlock() </code></pre> <p>And reading with</p> <pre><code>index.RLock() thing = index.m[idx] index.RUnlock() </code></pre></pre>

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:692541889

206 次点击  
加入收藏 微博
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传