Anyone else find vendoring inconvenient for small projects?

So I have a bunch of small projects written in Go, most of which use more or less the same dependencies. Mainly a mysql driver, boltdb and a few packages which I wrote myself. The mysql driver and boltdb are both very stable projects. I don't think there was ever any case where I updated them and my code broke.

I've ended up realizing that in this case, vendoring seems to be giving me more trouble than it solves. I've decided to delete the vendor folder and go back to the simplicity of "go get" for now which seems to be getting the job done just fine.

Has anyone else tried this before? Have you regretted it? Should I go back to vendoring? Thanks.

---

评论：

sh41:

I've decided to delete the vendor folder and go back to the simplicity of "go get" for now which seems to be getting the job done just fine.

I expect this not to be a popular preference, but I've been doing that and really enjoying it. I use latest versions of the vast majority of my dependencies (I update them regularly with Go Package Store) without vendoring.

However, most of my dependencies are:


my own packages or packages I help maintain (have push rights to)

are created by people I trust to maintain them well

are mature and don't change APIs often, and when they do, it's for the better and easy to update

In the end, without vendoring, if you pick your dependencies with care, it's very easy to have 95%+ uptime. But if you want 100% uptime, then you have no choice but to vendor.

Vendoring feels like a very expensive solution that makes sense to use when it's unavoidable. In all other cases, it's so much simpler and more lightweight to skip it, since it really only buys you at most 5%~ uptime.

bupku5:

you have given a perfectly rational explanation of how you develop safely with vendoring. you didn't insult or misquote anyone. yet you are still downvoted in this sub because you are apparently violating some zeitgeist

/r/golang is really becoming terrible

tv64738:

Downvoted? It's the highest-rated comment..

ChristophBerger:

There is always someone who downvotes posts or comments for no good reason. As I am writing this, /u/sh41's comment has 19 upvotes, and this clearly shows that knee-jerk downvotes are not a typical behavior in /r/golang.

ragefacesmirk:

It's easier to get up votes for causing a riot... It's not just here

bupku5:

if you aren't careful, vendoring can be far more misguided than just using go get

most people don't bother rationalizing the version they fixate on, for all they know they are fixating on a version the author regretted and changed ten minutes later

you need to have the discipline to regularly check your dependencies for version changes and bugfixes, no different than if you used go get, otherwise you may be just fixating on obsolete garbage

adamtanner:

Relying on the GOPATH and go get is basically how Go at Google works.

The GOPATH is modeled after the way Google does source control as a massive monorepo. In that world the GOPATH is the root of your repo so everything is checked in and essentially vendored already. 

This works really well if you have a bunch of closed source projects that share the same dependencies and you don't mind checking out the entire world or doing fun git tricks to select subtrees.

I've found the most successful cases of this tend to be with multiple teams/projects with shared dependencies like auth, logging, API clients, etc. Version skew sucks, but so does having to "update the whole world", so it's a trade-off. :)

Vendoring on a per project basis tends to work really well for self-contained code bases, like most open source projects whose output are executables, that you intend for others to checkout and build. It is generally not recommended to vendor inside of packages that are intended to be used as libraries by others.

kaeshiwaza:

Which trouble do you have using dep (specially for small project) ?

nesigma:

There's no trouble. I just find "go get" more convenient for the case I've described above (small projects with few and stable dependencies).

kaeshiwaza:

The answer of your question is your first sentence. Why loose the ability to know which revision of your dependencies you used when it's so easy ? It's a kind of thing that it's too late when you find out you needed it...

nesigma:

As I said, I find it inconvenient for the case I described above. Sure without dep I am going to lose the ability to know which revision of my dependencies I used. But that's the thing. I never had to think about this before. By default with "go get" I have to always be on a working version. There's nothing extra to think about. It either works or it doesn't.

Also with dep or another tool, unless you vendor your dependencies, then people have to do an extra command to build the project. And if the dependencies are vendored then you end up with big vendor folders. For example just the 2 dependencies I described above (mysql + boltdb) make a vendor folder of around 5.5MB. 

None of this is especially troublesome but for a small project with few and stable dependencies I find that the simplicity of "go get" goes away with those little problems and extra overhead.

Anyways, based on the current answers it seems that I am not the only one that does this. I hope to see more answers and opinions but the current answers have encouraged me to keep doing it.

kemitche:

There's a potential middle ground - using dep or glide or similar to write a vendor.json or similar file with the "last known good" dependency version, but not requiring the use of those tools to populate the vendor/ dir after fetching the repo.

nesigma:

but not requiring the use of those tools to populate the vendor/ dir after fetching the repo

How can this be done?

kemitche:

Basically, go get doesn't know about any of the vendoring helper tools. It only knows that, if a package is in vendor/, to use that instead of $GOPATH/src for that package.

The tools can usually work in one of two ways.


You commit the vendor/ directory to source control. In this case, the tools just assist with figuring out what packages to get, and getting the right version of them into vendor/. During build time, the tool itself is unnecessary; go get will "just work."

You DON'T commit the vendor/ directory. Instead, you just commit the package.json or gopkg.toml or equivalent file; that file just says "we want to build this using the dependencies in this file at the versions listed in the file." In this case, you're instructed NOT to go get the package or command; instead, you fetch the source "normally" (or use go get -d to download, but not build). Then, you run glide or dep or govendor or whatever to locally create (but again, not commit) the vendor/ directory, fetching all appropriate packages.

My suggestion - and I haven't tried it myself - would be to go with option 2. Don't commit the vendor/ directory. Then just tell people that go get should work. What should happen is that go get - ignorant of the gopkg.toml (or whatever file) goes through the normal process of fetching dependencies into $GOPATH/src. Anyone who encounters issues can use glide (or dep or govendor) to try rebuilding with the "last known good" dependencies.

ChristophBerger:

So basically, what you suggest is always using the latest dependencies via simple go get (i.e., no vendoring of any kind) but at the same time keep track of the last known good version of each dependency, to be able to revert to that version in case building or testing wit the latest version fails.

This is indeed an interesting idea. Do the current dependency tools support this kind of "version bookkeeping without vendoring" out of the box?

kemitche:

This is indeed an interesting idea. Do the current dependency tools support this kind of "version bookkeeping without vendoring" out of the box?

I wrote more about it as a reply to the other comment: https://www.reddit.com/r/golang/comments/6m7vic/anyone_else_find_vendoring_inconvenient_for_small/dk0kbov/?context=3

I'm only speculating based on how I've seen the tools work. I think they'll all work this way out of the box. Since go get is completely oblivious to the tools, if you don't commit vendor/, just the package manifest, and never run the 3rd party tool, go get should use $GOPATH/src (including fetching any missing dependencies, at their latest commit).

ChristophBerger:

and never run the 3rd party tool

(...except for updating the last known good version after all tests succeed)

That's the step (or rather, non-step) that was missing in my mental model of your approach. Thanks. 

xchapter7x:

I'd suggest taking a look at glide (https://github.com/Masterminds/glide)
You can pin versions, or pull latest. No need to keep a vendor dir committed, just a yaml defining your deps.

There is also the emerging standard (https://github.com/golang/dep) which takes a similar approach

nesigma:

I am already using dep for some projects. I find "go get" more convenient for the case I've described above (small projects with few and stable dependencies).

lobster_johnson:

Keep in mind that Glide is horribly, horribly buggy. I've been using it for more than a year, and when it works, it's fine, but mostly it just craps out or does the wrong thing. In fact, the released version (0.12.3) doesn't even work on most of our repos; we have to use the development version, otherwise it randomly fails with inexplicable errors such as "could not detect VCS" or HTTP 404 (on packages that do exist).

These issues have persisted across a number of releases, and even the authors seem to agree about how awful it is. If you look through the Github issues, many of them seem to be waiting for some kind of new "versioning engine" that will clean everything up. But this seems to be a long-running project that isn't ready yet.

I'm really looking forward to replacing it with Dep.