<p>Hi guys, right now I'm using redis and globs with the KEYS command for found files in a big directory structure, so</p>
<p>KEYS <em>cat.jpeg</em>
returns /somefolder/somepath/blah/cat.jpeg</p>
<p>this works great and it's pretty fast but it has 2 problems: 1) I need use redis and I'd prefer some embedded alternative where the user don't need install some external libraries
2) redis optionally persist to disk but this keep the data in memory, if my data is 500 mb, that is a lot the ram wasted (for a small app)</p>
<p>I know than there are many golang key value db and I think than even with plain go and without use a full db (persisting to file and reading it as stream) would be ok</p>
<p>recently I watched this post
<a href="https://www.reddit.com/r/golang/comments/55pyas/how_to_read_2gb_text_file_with_small_ram/" rel="nofollow">https://www.reddit.com/r/golang/comments/55pyas/how_to_read_2gb_text_file_with_small_ram/</a></p>
<p>and I think than would be very suitable achieve this with a similar solution, but I'd like ear recomendations, which solution could you consider the most appropriate for this task? any recomendation for achieve this with a nice performance? I think than the file could not be so big, just 512 mb or so</p>
<p>thank you guys </p>
<hr/>**评论:**<br/><br/>Justinsaccount: <pre><p>If the point is to find files based on their base names you should just store files using a key of</p>
<pre><code>basename:full path
</code></pre>
<p>then you can just do a prefix search for <code>basename</code></p></pre>dsidhion: <pre><p>Have you seen <a href="https://github.com/boltdb/bolt" rel="nofollow">boltdb</a>? Looks like what you want, and it uses memory-mapped files, so the OS will be the one taking care of loading as much of a file as it can.</p></pre>lalolambda3141592: <pre><p>thanks for the reply, yes I check boltdb but I'm not sure if it supports globs/regex for get keys.although I think than would be possible use a combination of iteration and regex... thanks</p></pre>captncraig: <pre><p>I've used ledisdb as a sort of embedded redis option. We can start a server in-proc on a high port and use a real redis client to talk to it. Then users have an embedded option, but can graduate to dedicated redis if they wish. </p>
<p>It works well, with only a few gotchas. </p></pre>xiegeo: <pre><p>Would SQLite suffice? </p>
<p><a href="https://www.sqlite.org/whentouse.html" rel="nofollow">https://www.sqlite.org/whentouse.html</a></p>
<p><a href="https://github.com/mattn/go-sqlite3" rel="nofollow">https://github.com/mattn/go-sqlite3</a></p></pre>
这是一个分享于 的资源,其中的信息可能已经有所发展或是发生改变。
入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:692541889
- 请尽量让自己的回复能够对别人有帮助
- 支持 Markdown 格式, **粗体**、~~删除线~~、
`单行代码`
- 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
- 图片支持拖拽、截图粘贴等方式上传