<p>Hello!
I am currently working on a web server written in Go, which returns HTML pages or images, depending on the requested URL. Now I started reading about whether to store images in the file system or as a blob in the database and found out, that just storing image paths in the database was way easier, more efficient and probably even cheaper. So I thought about that for a little while and just had the idea to write an external Go/C++ server just for serving images to the web server, which would enable the image servers to run on multiple host devices, with the correct IP addresses of the hosts and the image's paths stored in the database. That way, if the images were moved for example, I'd just have to change the code of the image server to look in another directory instead of having to update all the paths stored in the database because the database would just hold something like 192.168.1.15/img/testimage.jpg and the image server would return it.</p>
<p>Does this make sense or is there a better solution to this? And how would I implement such image servers and the communication between the image servers and the web server? Would I be better off using something like FastCGI, or good old HTTP/HTTPS?</p>
<p>Thanks in advance for taking the time to read and answer this :)</p>
<hr/>**评论：**<br/><br/>tlianza: <pre><p>I think if you have a server serving images to another server - ie you're passing images between servers - you're probably doing something wrong.</p>
<p>A database table with paths is fine. In fact, you might split hostname/root path from the file name, to give you some flexibility on moving images around. But, service-wise what you're describing sounds like it has more moving parts than you'd likely need.</p></pre>mko31: <pre><p>Thank you for taking the time to answer this.</p>
<p>So you think I could just store the host address in a column, the path or file name in another one and then serve the images on given host with what? nginx, like <a href="/u/google_you" rel="nofollow">/u/google_you</a> suggested?</p></pre>QThellimist: <pre><p>Amazon S3 is a viable solution. You'd store the images there and save the url's. Also it can support multiple intances(web severs).</p></pre>mko31: <pre><p>Many people suggested Amazon S3 to me already, but I don't really want to rely on external services. I'm still in school and have time to learn to write things like this myself (I guess).</p></pre>QThellimist: <pre><p>just don't save the images in the database. use the filesystem or an external file system for your use case.</p></pre>TheMerovius: <pre><p>Why do you want to save anything in the database at all? Have you considered using the filesystem for that (e.g. have a directory for your images and some ASCII-id as a filename that you use for additional information). The advantage of that is not only simplicity, but also that http.FileServer will give you the correct caching-header, range requests and Content-Type header for free.</p>
<p>You might be able to find a distributed filesystem that you can just fuse-mount to distribute and reduntantly store the files, then you wouldn't have to worry about the server they're on at all. :)</p>
<p>Even if you go by your database design, I would still suggest implementing an http.FileSystem on top of that for the aforementioned benefits. But writing distributed filesystems yourself (which is more or less what you are about to do) is actually pretty hard, if you want to do it right. For example: You mention moving images. How do you ensure, that the database is always consistent with where your images really are stored? What if the server where an image is stored suddenly fails? What is your redundancy-strategy and how do you map that to your database schema? How do you do failover?</p>
<p>I suggest choosing the very simplest solution you can, at least at the beginning. The simpler a system is, the harder it's to do it wrong, the easier it is to maintain and the more reliable it will be.</p></pre>mko31: <pre><p>I want to use a database because I need to store additional information like a title (up to 100 characters), a description (up to 400), who uploaded it and all its comments.</p>
<p>But <a href="/u/google_you" rel="nofollow">/u/google_you</a> and <a href="/u/tlianza" rel="nofollow">/u/tlianza</a> might be right, just as you. I'd probably be better off using Apache or nginx at least for the image server.</p>
<p>And as I said to <a href="/u/google_you" rel="nofollow">/u/google_you</a>:
"I really don't just want to fetch them from the file system because the service is supposed to offer and share thousands of images between people who can upload them, etc."</p></pre>TheMerovius: <pre><p>Believe me, when I say, that Google and Facebook and Imgur… also just use a distributed filesystem as their image storage. I think, using the filesystem (or rather: a distributed filesystem) is indeed a <em>lot</em> more scalable than saving images in a database.</p></pre>mko31: <pre><p>But how would I retrieve an image without storing the location somewhere?</p></pre>calebdoxsey: <pre><p>If you use S3 (a distributed file system) you can upload the images with a tool like cyberduck or build an admin section and use Go to upload the images.</p>
<p>You could use a UUID for the image file name in S3. Or a SHA1 hash of its contents. (which might be useful for de-duping)</p>
<p>S3 would give you a public URL (assuming you want that) which you would store in your database. Then you don't need an image server at all because the images would be coming from S3.</p>
<p>S3 is cheap, fast and very durable. Its uptime is probably better than you could manage to achieve on your own. It also has sophisticated CDN capabilities which you can't pull off with a single server.</p>
<p>Google Cloud Storage is basically equivalent to S3 if you want another option.</p></pre>google_you: <pre><p>There are plenty of web servers that can serve html and images. Why create your own? Use Apache HTTPd or NGiNX?</p></pre>mko31: <pre><p>I want to create the web server myself, but I could use nginx or Apache to get the images from external machines, right?
I really don't just want to fetch them from the file system because the service is supposed to offer and share thousands of images between people who can upload them, etc.</p></pre>
