<p>I have a function that grabs an asset from a DB and places it in an in memory cache, and I'm trying to solve an issue where if multiple requests for the same asset are made simultaneously, the 'second' request needs to wait until the first request is done processing and has placed the asset into memory so both requests don't query the DB.</p>
<p>I've been messing with <code>sync.RWMutex</code>'s and they seem to be working, but when running the application with the <code>--race</code> flag it's warning me that there are race conditions, even though I haven't ran into any while attempting to simulate a production environment.</p>
<p>Here's a basic example of what I am running:</p>
<pre><code>var assetSync = make(map[string]*sync.RWMutex)
func GetAsset(id string) (asset Asset) {
var as sync.RWMutex // Using this suppressed some warnings being thrown about concurrent read/writes
if _, ok := assetSync[id]; !ok {
as.Lock()
assetSync[id] = &sync.RWMutex{} // Make new mutex
as.Unlock()
}
as.Lock()
assetSync[id].Lock() // Lock the mutex so the next request for the ID is forced to wait here
as.Unlock()
if x, found := GC.Get(id); found {
// Found in cache
asset = x.(Asset)
} else {
// Going to query DB
asset = GetAssetFromDB(id)
GC.Set(id, asset)
}
as.Lock()
assetSync[id].Unlock() // Unlock the mutex for the ID
as.Unlock()
}
</code></pre>
<p>The compiler is complaning that different go routines can write to the assetSync hash map, but that is the point and I'm OK with it but obviously I'd like to do it in a way that is thread safe.</p>
<p>Any ideas?</p>
<p>EDIT:</p>
<p>I was able to come up with a solution that isn't throwing race conditions anymore thanks to suggestions from bwandel. The updated, simplified code would look like the following:</p>
<pre><code>func getAccessLock(key string) *sync.RWMutex {
if lock, found := GC.Get("access_lock"+key); found {
return lock.(*sync.RWMutex)
}
lock := &sync.RWMutex{}
GC.Set("access_lock"+key, lock, cache.NoExpiration)
return lock
}
func GetAsset(id string) (asset Asset) {
lock := getAccessLock(id)
lock.Lock()
defer lock.Unlock()
if x, found := GC.Get(id); found {
// Found in cache
asset = x.(Asset)
} else {
// Going to query DB
asset = GetAssetFromDB(id)
GC.Set(id, asset)
}
}
</code></pre>
<p>GC is a global cached I already use powered by: <a href="https://github.com/patrickmn/go-cache">https://github.com/patrickmn/go-cache</a></p>
<hr/>**评论：**<br/><br/>bwandel: <pre><p>there is a lot wrong with this code, but as to your specific question: looks like you are not sharing "as" between calls... that is, your assetSync sets/gets are not done under mutual exclusion.</p>
<p>example: <a href="https://play.golang.com/p/Ua7gjQnbAf" rel="nofollow">https://play.golang.com/p/Ua7gjQnbAf</a>
note I've moved "var as ...." to line 8.</p>
<p>by the looks of the code, you haven't done a lot of concurrent programming, so I would encourage you to experiment with it. Its one of those skills that are invaluable.</p>
<p>edit: here is an example of some better code. not perfect, but focuses on the locking for you. there is also sync.Map.
<a href="https://play.golang.com/p/SrHbuCTJtr" rel="nofollow">https://play.golang.com/p/SrHbuCTJtr</a></p></pre>dahlma: <pre><p>Thank you, I appreciate the feedback. I will test the second example you sent and see if that works. I agree the code is a mess, it’s based off code that’s a few years old and it’ll eventually be rewritten but as you can imagine a lot of the other code looks similar in style and also needs to be rewritten, so it’s a slow process.</p>
<p>I initially did what you suggested in your first example, but when you run parallel requests to that the app essentially locks up because now all requests are waiting for the slowest request, even if another asset is a lot quicker to access than another.</p></pre>RenThraysk: <pre><p>slngleflight</p>
<p><a href="https://github.com/golang/groupcache/blob/master/singleflight/singleflight.go" rel="nofollow">https://github.com/golang/groupcache/blob/master/singleflight/singleflight.go</a></p></pre>twetewat: <pre><p>I'm very interested in this aswell.
The only example i've come across is this <a href="https://github.com/go-chi/httpcoala" rel="nofollow">https://github.com/go-chi/httpcoala</a></p></pre>
