Can I get some help using httptest.Server

As the title suggests I'm using http/httptest.Server to test a small app I am writing to help me learn Go.

The project can be found here: YAHNCrawler. It prints the story information of the top stories on Hacker News. I'm striving towards concurrency here.

My problem, getting the testServer to handle different url paths:

my query code:

// query performs a HTTP: GET request to the specified url.
// Storing the response's body in the `result`.
// query expects the provided `result` to conform to the query's repsonse body.
func query(url string, result interface{}) *interface{} {
    resp, err := http.Get(url)
    check(err)
    body, err := ioutil.ReadAll(resp.Body)
    resp.Body.Close()
    check(err)

    unmarshalErr := json.Unmarshal(body, result)
    check(unmarshalErr)

    return &result
}

The issue is this method is called twice. Once to get a list of ids of the top stories, the next call queries the information of a story by one of these ids using a different url path.

I have tests for each call but I want to do a full end-to-end test of the exported function CrawlTopStories.

my test:

func TestCrawlTopStories(t *testing.T) {
    testServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, `[ 14223020 ]`)
    }))
    defer testServer.Close()

    CrawlTopStories(testServer.URL)

            // TODO fail test if not passed
}

Above in TestCrawlTopStories I have set up the test server to respond with the id of a single story, necessary for the first call. But I need a separate handler to support the second call to "/story/{id}".

Is there a solution where I can reuse the same testServer? Reading httptest.Server.go hasn't shed any light on this issue, maybe I need to look deeper.

Once I have this solved I would like to address this code:

storyChan := make(chan *Story, topStoryCount)
go func(storyCh chan<- *Story) {
    for _, topStoryID := range *topStoryIDs {
        storyCh <- queryStory("https://hacker-news.firebaseio.com/v0/item/" + strconv.Itoa(topStoryID) + ".json?print=pretty")
       time.Sleep(500 * time.Millisecond)
    }
    close(storyChan)
}(storyChan)
....

Ideally I could make each call, concurrently in a separate goroutine, but then I wouldn't be able to safely close the channel before ranging over it. As I will be querying 500 stories, is there a point in trying to improve this?

Many thanks

So you need routing for your httptest server.

Well it's exactly the same as routing for a http server.

For a quick solution, you can use http.ServeMux. Build one, and pass that to httptest.NewServer instead of your http.HandlerFunc.

Ultimately, if your test data are 100% static, I think you would be better off storing them in files and using a http.FileServer

Thanks! I'll look into this. A problem I've found with Go is that due to its implicit inheritance, it's hard to find implementations of interfaces and therefore would likely not have found ServeMux for a long time. Do you know of any method of finding existing  implementations?

Go Guru