<p>Hello!</p>
<p>So, I have this project I've been working for a few months now. Here is one concern I have with it: benchmarking.</p>
<p>I wrote an integration test, not large enough (read: covering the entire application) thought, which happen to be a benchmark test too, here is the code:</p>
<pre><code>package main
import (
"log"
"testing"
"github.com/go-telegram-bot-api/telegram-bot-api"
mgo "gopkg.in/mgo.v2"
redis "gopkg.in/redis.v4"
)
// Borra la base de datos y limpia Redis
func teardown() {
config := LoadConfig()
// MongoDB
mongo, err := mgo.Dial(config.DB.Address)
if err != nil {
log.Fatalf("No se pudo conectar a MongoDB: %s", err)
}
defer mongo.Close()
err = mongo.DB(config.DB.Name).DropDatabase()
if err != nil {
log.Fatalf("No se pudo borrar la base de datos: %s", err)
}
// Redis
redis := redis.NewClient(&redis.Options{
Addr: config.Redis.Address,
Password: config.Redis.Password,
DB: config.Redis.DB,
})
err = redis.FlushDb().Err()
if err != nil {
log.Fatalf("No se pudo borrar la base de datos Redis: %s", err)
}
}
func BenchmarkReaction(b *testing.B) {
defer teardown()
// preparamos al bot
config := LoadConfig()
fe, err := NewFrontend(config)
if err != nil {
log.Fatalf("No se pudo crear Frontend para pruebas de integración: %s", err)
}
fe.RegisterSessions(
&MainMenu{name: "main menu"},
&SelectLanguage{name: "select language"},
)
// canal de actualizaciones
sendupdates := make(chan tgbotapi.Update)
// canal para recibir respuestas
answers := make(chan tgbotapi.Chattable)
go func() {
for {
select {
case <-answers:
// hacemos nada, porque no nos interesa el
// mensaje de vuelta
}
}
}()
// Iniciamos el bot
go fe.Listen(sendupdates, answers)
for n := 0; n < b.N; n++ {
// presentación
sendupdates <- tgbotapi.Update{
Message: &tgbotapi.Message{
From: &tgbotapi.User{
ID: n,
FirstName: "Dummy",
LastName: "MacDummy",
UserName: "DD",
},
Text: "/start",
},
}
// Registro luego de escoger el nombre de usuario
sendupdates <- tgbotapi.Update{
Message: &tgbotapi.Message{
From: &tgbotapi.User{
ID: n,
FirstName: "Dummy",
LastName: "MacDummy",
UserName: "DD",
},
Text: "Español",
},
}
}
log.Printf("Registrado %d usuarios", b.N)
}
</code></pre>
<p>The thing here tested is the first two interactions of the user with the app, he is registered in the first interaction (this is a Telegram Messenger bot) and by the second intereaction some data is retrieved from the database (What menu should be shown to the user) and the file system (Internationalization). Some stuff is cached on first read using Redis too.</p>
<p>Anyway, I ran the benchmark a couple of times and these are the results:</p>
<pre><code>→ go test -bench=.
PASS
BenchmarkReaction-2 2016/08/23 19:27:34 Roles y actividades registrados en la base de datos
2016/08/23 19:27:34 Registrado 1 usuarios
2016/08/23 19:27:35 Roles y actividades registrados en la base de datos
2016/08/23 19:27:35 Registrado 2 usuarios
# [···]
2016/08/23 19:27:39 Registrado 50 usuarios
2016/08/23 19:27:39 Roles y actividades registrados en la base de datos
2016/08/23 19:27:40 Registrado 100 usuarios
100 10514644 ns/op
ok bitbucket.org/capslockdev/betsy 6.344s
→ go test -bench=.
PASS
BenchmarkReaction-2 2016/08/23 19:28:31 Roles y actividades registrados en la base de datos
2016/08/23 19:28:31 Registrado 1 usuarios
1 2209585800 ns/op
ok bitbucket.org/capslockdev/betsy 2.350s
</code></pre>
<p>And, I'm very worried that registering and interacting with 100 users (right? that's how I should read it?) takes 0.00010514644 seconds for each user, or that it later took 2.2095858 seconds to register and interact with just ONE user...</p>
<p>Well, actually what buggies me is the fact that I don't know if these numbers are something to call "slow", I added the cache for my MongoDB before writing this benchmark, and I also don't feel like trusting these numbers I get from the benchmark, the sudden change from 100 to 1 and the time differences gave trust issues. I'm testing in a Thinkpad T60 running GNU/Linux, it should be an acceptable environment for benchmarking I guess.</p>
<p>So, to wrap this here:
* am I doing this benchmarking thing right?
* Is there a way to get more information than the time it took for the app to complete the test?
* Exists any way to get "control" data to compare my benchmark, besides removing some performance-saving feature from my project (like the DB cache) and running the benchmark again?</p>
