<p>Hi All,</p>
<p>I want to write a basic web browser in Go. The reason for doing this would be two fold. Firstly, to learn a bit about computer networking such as HTTP, TCP/IP and socket programming. Secondly, I want to learn a bit of go. I figured that since go was created with low level computer networking in mind that it would be the perfect language to use. Just a disclaimer, this is a school project but I am not asking any one to write any code for me - I'm simply coming to you all for advice :) Basically before I get stuck in I want to find out if Go will in fact be suitable, most of my class is going to be using Python for the project so I don't want to be digging my self in a hole I can't get out of :)</p>
<p>So here is where I am. I have played around with socket programming in Go, using the net package (<a href="https://golang.org/pkg/net/" rel="nofollow">https://golang.org/pkg/net/</a>) in the standard library. This was very straightforward and thanks to go routines making it concurrent was blissful! In fact the library was very similar to the Python socket library (<a href="https://docs.python.org/3/library/socket.html" rel="nofollow">https://docs.python.org/3/library/socket.html</a>), which I have played with a little. What I am unsure of is this: will I be able to (without too much trouble) make a web browser GUI. I am unsure of the best GUI libraries for Go and their capabilities. What I am most unsure of though is if there is a library for rendering HTML. This is not the focus of the project and as I understand it is a very complicated process so I don't want to be spending all my time trying to figure that out. The main focus of the project is to implement HTTP using socket programming. </p>
<p>So I guess my questions for you all are:
1. Is this feasible?
2. If so what libraries should I look at for HTML rendering and creating the web browser GUI?</p>
<p>Just to clarify, this web browser doesn't have to be able to display every page on the internet - just a few like the Google and Wikipedia home pages, as well as a few on a web sever I will also be writing.</p>
<p>Thanks for the help! </p>
<hr/>**评论：**<br/><br/>Ainar-G: <pre><p>I've played with the idea of a browser in Go actually. I'd suggest going through <a href="https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.html" rel="nofollow">Let's Build a Browser Engine!</a> and trying to hack something with <a href="https://godoc.org/golang.org/x/net/html" rel="nofollow"><code>golang.org/x/net/html</code></a> for html parsing and <a href="https://godoc.org/golang.org/x/exp/shiny" rel="nofollow"><code>golang.org/x/exp/shiny</code></a> for GUI. Shiny is a WIP so be prepared to some API changes.</p></pre>Jimbo_029: <pre><p>Thanks those are some really good references. I'll definitely take a look at that toy web browser engine. </p></pre>mwholt: <pre><p>The GUI would be the hardest part; first using the native window server, then rendering the graphical elements on the page properly. Of course, you could use a C library like WebKit or something to do the rendering, but cgo isn't Go.</p>
<p>Far more feasible with Go would be a command-line browser like Lynx. All you have to do (well, basically) is a GET request, then pick the HTML elements you can display in the console. It's much simpler this way. The ultimate challenge here would be laying out the content in the console in a way that's meaningful and interactive, but I bet there are ways to do it like Lynx or even reinvent that a little bit.</p></pre>bmatsuo: <pre><p>Saying that "cgo isn't Go" seems a little unfair. I understand it isn't pure and in that sense the development experience is not ideal. Seriously though, we are really just trying to build cool software.</p>
<p>Isn't a modern graphical browser going to be using OpenGL anyway (maybe Vulkan for the super modern :p)? If you are using either OpenGL or Vulkan then cgo cannot be avoided. No?</p>
<p>Honestly it may be quite difficult to work with WebKit in Go. I have no idea how the API is structured. And, as I understand it can be difficult working with some C application frameworks. I just think disregarding everything as impure is a little overzealous in this case.</p>
<p>edit: Worth noting that even the runtime can suffer from using cgo. And it is worth avoiding when <em>possible</em>.</p></pre>mwholt: <pre><blockquote>
<p>Saying that "cgo isn't Go" seems a little unfair.</p>
</blockquote>
<p>I didn't say it, Rob Pike did. <a href="http://go-proverbs.github.io/" rel="nofollow">http://go-proverbs.github.io/</a></p></pre>bmatsuo: <pre><p>Hah. Okay. Well I don't entirely agree with him there either. That attitude is pretty exclusionary. As I said, sometimes you really can't avoid it. I don't think that population of users should be marginalized.</p>
<p>I hadn't seen that site though. Thanks for the link.</p></pre>Jimbo_029: <pre><p>Could you explain this cgo vs Go argument? Why should or shouldn't cgo be considered to be Go?</p></pre>lolomfgkthxbai: <pre><p><a href="http://dave.cheney.net/2016/01/18/cgo-is-not-go" rel="nofollow">http://dave.cheney.net/2016/01/18/cgo-is-not-go</a></p>
<p>Cgo eschews many of the things that make go good. Using cgo your go application is at worst just a go wrapper around a C application.</p></pre>int32_t: <pre><p>One can implement the user space part of the graphic stack (like Mesa on Linux) as a Go package which talks to the kernel/device drivers directly with system calls.</p></pre>Jimbo_029: <pre><p>Hmmm, that's what I was worried about. I'll have a look at maybe using another language for the rendering. I'll also have a look at Lynx and see if I can swing that by my lecturer, thanks for the swift reply :)</p></pre>dmikalova: <pre><p>Also I think Qt has a built in webview, you could probably pretty easily hack together something that could be called a browser with that.</p></pre>j_heg: <pre><p>Which is not the same thing as writing a web browser in Go, though.</p></pre>weberc2: <pre><blockquote>
<p>cgo isn't Go</p>
</blockquote>
<p>This isn't meaningful in this conversation, considering the alternative is Python which will have the same dilemma as Go. At some point, you're going to have to interface with C irrespective of language choice.</p></pre>kor_the_fiend: <pre><p>If the main focus is to implement HTTP using socket programming, why not start with something much simpler, like an http server? Go is absolutely GREAT for this purpose!</p></pre>Jimbo_029: <pre><p>The whole project involves creating an HTTP server, a proxy server and a client using socket programming. I was drawn to Go for the former two but the later on is whats worrying me.</p></pre>kor_the_fiend: <pre><p>A client can mean a lot of different things - a browser being on the extreme end in terms of complexity. For example, you could write a client that queries a restful api (maybe the HTTP server you wrote for this same project) based on user interaction or some other input, then renders the data in some meaningful way (without trying to render the entire DOM). </p></pre>Jimbo_029: <pre><p>A think that's a great point - what I may do is just implement a simplified client that just makes http requests and dumps the HTML to the console. I might loose some marks but I think that the main point of the project isn't to make a web browser so as long as I show that I understand HTTP I should be mostly fine :) </p></pre>SupersonicSpitfire: <pre><p>Qt could be used for the GUI. Porting Netsurf (written in C) to Go is an option.</p></pre>carleeto: <pre><p>If the focus is really the networking stack, then check with your professor if you could write a web crawler instead.</p>
<p>You would still have to specify a URL you would still be parsing html and you would still be making connections.</p>
<p>In addition, you could have fun with goroutines and showcase what easy concurrency combined with robust networking could achieve.</p>
<p>If the focus isn't the GUI, it should turn mundane questions like "how do I render lots of text on the screen" into "let's explore the effect of concurrent HTTP requests on a web server"... which is much more fun :-)</p></pre>Jimbo_029: <pre><p>The lecturer was pretty specific about wanting the web browser and GUI, but I think I will just scale it down to be a command line "browser" that just dumps the HTML to the console. I fell that the focus shouldn't be on making GUIs and rendering HTML, but rather understanding and implementing HTTP.</p></pre>dericofilho: <pre><p>Although Cgo is not Go, it doesn't mean that all your code must be tainted with Cgo. Chrome, for instance, uses RPC and separated processes for several architectural reasons. </p>
<p>So you can actually use Webkit renderer through Cgo, but wrote everything else in pure Go and connecting each component through some sort of IPC. </p></pre>daddyc00l: <pre><p>if the goal is to learn about networking etc then a 'simple' text based browser e.g. lynx might be a better goal. heck just try and mimick the 'lynx --dump <url>' for starters :)</p></pre>Jimbo_029: <pre><p>Thanks, I think that is definitely the way to go with this if I am going to use Go.</p></pre>lonahex: <pre><p>I would suggest to ditch the browser and implement a library and in Go that lets other developers use it for common HTTP operations. Much like Python's requests library. It would be best suited for this learning project. Once you are done with this and if you still feel passionate, you can start working on a GUI that fetches and renders webpages using the library you wrote but it would be completely different projects and most work would be around GUI and writing parsers/lexers, etc depending on how much you want to implement yourself. </p></pre>Jimbo_029: <pre><p>I'm definitely considering dropping the web client. But I think I'm just going to focus on building web and proxy servers. Though your suggestion may be a good idea for the future - thanks :)</p></pre>krzykizza: <pre><p>There is a class that i do not personally attend to that is done purely in go. Their next assignment is proxy server, http server, and a client. I have worked with go a bit, and i think its great for this kind of stuff, the hardest part would be the user interface, but getting to the point where you can say, fill a buffer with whatever you get through http should be fairly simple.</p></pre>dandanican: <pre><p>Another contemporary is Rust programming language. </p></pre>Jimbo_029: <pre><p>I did consider that. It's defiantly something that I want to try out in the future. But you know one step at a time :)</p></pre>SupersonicSpitfire: <pre><p>Rust is nice if you want to think a lot about memory, for optimization reasons, while don't minding using the full ASCII table when programming. No special character shall go unused!</p></pre>Jimbo_029: <pre><p>That is actually why I chose not to use rust for this. Don't get me wrong, I enjoy my manual memory management and C++. However, for this project I wanted to be focusing on HTTP and socket programming without getting lost in the memory management. Rust is definitely something I want to check out though and I am very interested in systems programming.</p></pre>: <pre><p>[deleted]</p></pre>Jimbo_029: <pre><p>I am doing that too, but unfortunately my lecturer wants both. I'm considering writing the web browser in Python and then the server in Go</p></pre>dyoo: <pre><p><a href="http://grail.sourceforge.net/" rel="nofollow">http://grail.sourceforge.net/</a></p>
<p>This was a web browser written in Python, a long, long time ago.</p></pre>earthboundkid: <pre><p>Frankly, it's an idiotic assignment. It takes about five years to write a web browser from scratch and about ten minutes to wire up a view using the system web browser. There's no good "do it in a month for class" middle point here. Either you do something trivial, or it cannot be done. </p></pre>lapingvino: <pre><p>writing a web browser in beginning programming classes = learning how to reuse the trident/gecko/webkit/blink stuff and create an interface around it.</p></pre>Jimbo_029: <pre><p>I'm inclined to agree with you, I don't think we should have to build the browser. I feel we should be focusing on the server side programming. I am going to talk to my lecturer about it.</p></pre>lapingvino: <pre><p>I would guess you could for example build on this? <a href="https://github.com/headzoo/surf/tree/dev/browser" rel="nofollow">https://github.com/headzoo/surf/tree/dev/browser</a></p></pre>
