<p>Hi,</p>
<p>I have updated Minke recently (<a href="https://minke.rocks" rel="nofollow">https://minke.rocks</a>) to add a few new features, Minke allows you to scaffold and build a Go microservice only using bash and Docker. You do not even need Go installed which makes it a perfect workflow for CI. Minke is designed to cover the full circle of quality for microservices, it will build code, execute unit tests and can also run cucumber behavioural tests.</p>
<p>Minke can start a stack of services such as databases and consul which are all user defined with docker-compose, it also has the capability of shell access to a build container and forwarding SSH keys for private repositories.</p>
<p>I have been using this in production now for over a year and the latest release is really getting where I want it, other than Go it supports other languages with pluggable generators.</p>
<p>Check out the video and let me know what you think.</p>
<p>Simple example:<br/>
<a href="http://github.com/nicholasjackson/example" rel="nofollow">http://github.com/nicholasjackson/example</a><br/>
<a href="https://asciinema.org/a/105820?speed=2" rel="nofollow">https://asciinema.org/a/105820?speed=2</a></p>
<p>Complex example:<br/>
<a href="https://asciinema.org/a/105978?speed=2" rel="nofollow">https://asciinema.org/a/105978?speed=2</a><br/>
<a href="https://github.com/nicholasjackson/sorcery" rel="nofollow">https://github.com/nicholasjackson/sorcery</a></p>
<hr/>**评论：**<br/><br/>sethammons: <pre><p>Hi jacksonnic. Thanks for sharing. </p>
<p>I have some suggestions and opinions. First, for the video, it is painfully slow to watch. This led me to learn I can add <code>?speed=3</code> to any asciiinema.org video (neat). You don't really show much in the video. The tutorial page shows the same info and does so very quickly. At the end of the video, I did not learn anything about your project structure or why I would want to leverage it for a new microservice. Being on <a href="/r/golang" rel="nofollow">/r/golang</a>, I want to see the Go code. After going through your site, I see you have some service discovery going on. And the test at the end of the video shows you have a <code>/health</code> endpoint. Is that it? I can't find any information on how the newly set up project is structured. This is the part that interests me the most. Maybe I missed the point, and you are only interested in the Docker aspect of starting up a service?</p>
<p>Tests. I see you are leveraging Ruby and Cucumber/Gherkin. We had a team at work use Ruby for integration tests on a Go project. Later, those tests were ripped out and replaced with integration tests written in Go. Personally, I don't like Gherkin. To compare the test in the video to what I may have written in a Go integration test. The output from your video (asciinema is pretty neat, copy-paste):</p>
<pre><code>Feature: Health check
In order to ensure quality
As a user
I want to be able to test functionality of my API
Scenario: Health check returns ok # features/health.feature:10
Given I send and accept JSON # vendor/gems/cucumber-api-0.4/lib/cucumber-api/steps.rb:11
When I send a GET request to the api endpoint "/v1/health" # features/steps/http.rb:1
Then the response status should be "200" # vendor/gems/cucumber-api-0.4/lib/cucumber-api/steps.rb:130
And the JSON response should have key "status_message"
</code></pre>
<p>I vastly prefer:</p>
<pre><code>// TestHealthCheck verfies the functionality of the API
func TestHealthCheck(t *testing.T){
// start the service (on a new port each time so tests can parallelize easily)
// handle things like closing the resp.Body. Maybe instead of returning http.Response, you have a sanitized version of that returned.
resp, err := TestHelper.StartServer().Get("/v1/health")
if err != nil {
t.Fatal("unable to init the service: %v", err)
}
if got, want := resp.StatusCode, http.StatusOK; got != want{
t.Error("got status %d, want %d with GET %s", got, want, healthURL)
}
// could leverage the health check struct and unmarshal, allowing assertions against values instead of string checking
if !strings.Contains(string(body), "status_message") {
t.Errorf("did not find %s in:\n%s", "status_message", string(body))
}
}
</code></pre>
<p>In summary, I suggest showcasing the structure of the Go microservice. I'd consider having an advanced tutorial where you network together multiple microservices into a pipeline and show how all the things can map together. This is one thing that we had to build at work where if we had something to leverage, we may have got that up and going and more uniform between teams. Leverage the language of the service for writing the integration tests. I guess the argument against this is that you want to generate projects for different languages and wanted one true way to test them all. I'd counter that your $LANGUAGE developer would prefer to write their integration tests in that language.</p>
<p>Thanks again for sharing!</p></pre>jacksonnic: <pre><p>Hi,</p>
<p>Good call on the speed I will update the post, this was recorded in real time and the product of which would be a service which is suitable for production, however </p>
<p>In terms of the Go code it is as vanilla as you can get as I do not really want to propose a framework for writing the actual go code, this is far too diverse a topic and everyone has their preferences. The problem I am solving is that there needs to be an abstraction away from the developers machine when it comes to continuous delivery. If you are purely compiling Go code then generally this is not a problem however the Go code is only part of the picture when it comes to microservices and you have issues with needing to test integration, you need to make sure that the container is actually ok and that any dependencies that are running in the container are ok, etc.</p>
<p>With writing Go tests for handlers using httptest.Server you are not really gaining complete test coverage that is testing from the outside of a running docker container inside to the application. In fact I never use httptest.Server as the speed with which the tests run is quite slow so what I do is structure my tests like the following.</p>
<ol>
<li>Write unit tests for http.Handlers replacing dependencies for datastores, etc where needed</li>
<li>Write cucumber tests for testing the external contract and for testing the integration to other dependencies like databases</li>
</ol>
<p>Whilst I do not have code coverage for my routing and server setup with step one I gain this with step 2 and therefore do not need to cover it on a code level. The unit tests for handlers are exceptionally quick and simple to write and execute in milliseconds giving me a really quick feedback loop.</p>
<p>The other problem with only testing with httptest.Server is that what about the dependencies that my application is under, consider for example it needs a datastore which is running in another container the location of which is managed by the service discovery mechanism of Consul, If I am using consul and consul template and I am packaging this into my Docker container then I should test this too. Then I have setup and tear down of the data in that database in order to set the initial conditions for my tests.</p>
<p>So step 2 could be achieved with Go rather than Ruby however Minke is a cross platform build tool and supports more than this, coupled to this the tooling already built with ruby such as factory girl and active record for dealing with data models is excellent. BDD I think is also an excellent tool for bridging the gap between business and development the use of which I actively encourage. Now again back to Golang we could satisfy the BDD element by using GoDog written by the DataDog team. There are two problems with this if this code lives too closely to the source code then there is a risk that we use the data libraries that the main code is using to setup and teardown our data, the risk with that is that there could be bug in that library which only manifests itself when run in a production-ish environment and a test which is too close to it would miss this. The other problem is how do we spin up this stack? Running a local database on your laptop is a big no for me when it comes to testing, not at least for the problems of getting new developers up to speed with your project, if you then decide to use DockerCompose you have to manage the stack startup and initialization from within your go tests. This fast becomes brittle and difficult to manage from project to project.</p>
<p>What minke does is abstracts this, it attempts to create an environment for building and testing your code which is reproducible from developers machine to CI server, it also attempts to centralize this logic giving an easy upgrade path should a breaking dependency in the DockerAPI be introduced for example, all my services no matter what language they are written is use the same CI setup (minke build_image, minke cucumber). What it also does is encourage a workflow which is not just testing the code but testing the container the code will run in and the config that the container will use, these environmental tests are often missed in my experience and also often responsible for failed deployments. When build my code in minke, I have tested a production container, more often than not I am confident enough to deploy the built container straight to the production registry. </p>
<p>I have added an example repo for the code but from a Go perspective this is rather vanilla.</p>
<p><a href="http://github.com/nicholasjackson/example" rel="nofollow">http://github.com/nicholasjackson/example</a></p></pre>jacksonnic: <pre><p>Hi Seth, good point on the example project I have updated a toy project which I wrote last year which was an implementation of an event sourcing server in Go. Please do not judge me on the Dependency injection, I was playing with this as a technique but this is a little marmite in the Go world and not a technique I use anymore.</p>
<p>This example uses Redis as a queue store and Mongo for a dead letter queue. The event sourcing server allows clients to register for an event stream and other servers to send events. The cucumber tests in this example test the full integration between the server and the datastore and also tests sending messages to fake clients.</p>
<p>Complex example:<br/>
<a href="https://asciinema.org/a/105978?speed=2" rel="nofollow">https://asciinema.org/a/105978?speed=2</a><br/>
<a href="https://github.com/nicholasjackson/sorcery" rel="nofollow">https://github.com/nicholasjackson/sorcery</a></p></pre>
