are there some key,value db which persist to disk and allow globs?

blov · · 604 次点击    
这是一个分享于 的资源,其中的信息可能已经有所发展或是发生改变。
<p>Hi guys, right now I&#39;m using redis and globs with the KEYS command for found files in a big directory structure, so</p> <p>KEYS <em>cat.jpeg</em> returns /somefolder/somepath/blah/cat.jpeg</p> <p>this works great and it&#39;s pretty fast but it has 2 problems: 1) I need use redis and I&#39;d prefer some embedded alternative where the user don&#39;t need install some external libraries 2) redis optionally persist to disk but this keep the data in memory, if my data is 500 mb, that is a lot the ram wasted (for a small app)</p> <p>I know than there are many golang key value db and I think than even with plain go and without use a full db (persisting to file and reading it as stream) would be ok</p> <p>recently I watched this post <a href="https://www.reddit.com/r/golang/comments/55pyas/how_to_read_2gb_text_file_with_small_ram/" rel="nofollow">https://www.reddit.com/r/golang/comments/55pyas/how_to_read_2gb_text_file_with_small_ram/</a></p> <p>and I think than would be very suitable achieve this with a similar solution, but I&#39;d like ear recomendations, which solution could you consider the most appropriate for this task? any recomendation for achieve this with a nice performance? I think than the file could not be so big, just 512 mb or so</p> <p>thank you guys </p> <hr/>**评论:**<br/><br/>Justinsaccount: <pre><p>If the point is to find files based on their base names you should just store files using a key of</p> <pre><code>basename:full path </code></pre> <p>then you can just do a prefix search for <code>basename</code></p></pre>dsidhion: <pre><p>Have you seen <a href="https://github.com/boltdb/bolt" rel="nofollow">boltdb</a>? Looks like what you want, and it uses memory-mapped files, so the OS will be the one taking care of loading as much of a file as it can.</p></pre>lalolambda3141592: <pre><p>thanks for the reply, yes I check boltdb but I&#39;m not sure if it supports globs/regex for get keys.although I think than would be possible use a combination of iteration and regex... thanks</p></pre>captncraig: <pre><p>I&#39;ve used ledisdb as a sort of embedded redis option. We can start a server in-proc on a high port and use a real redis client to talk to it. Then users have an embedded option, but can graduate to dedicated redis if they wish. </p> <p>It works well, with only a few gotchas. </p></pre>xiegeo: <pre><p>Would SQLite suffice? </p> <p><a href="https://www.sqlite.org/whentouse.html" rel="nofollow">https://www.sqlite.org/whentouse.html</a></p> <p><a href="https://github.com/mattn/go-sqlite3" rel="nofollow">https://github.com/mattn/go-sqlite3</a></p></pre>

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:692541889

604 次点击  
加入收藏 微博
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传