<p>Hi everyone!</p>
<p>I'm currently working on a dependency manager in Go that alleviates a lot of the issues faced by other implementations (no standard version locking, no platform for publishing actual packages to, no command line tool that is integrated with it all).</p>
<p>Does this sound like it would be useful? I promise it's not just another poor attempt with the same "solution" to a very real problem. It is currently closed source, but the plan is to open source it early enough to get feedback on features as I'm developing them. One of the first things I'm going to be releasing is a service similar to gopkg.in, but it will actually allow you to specify any semantic version tag (v0.1.2, v0.2.1, v1.3.4, etc) for the repo endpoint. It will also support any public Git URL, so you won't be limited to GitHub.</p>
<p>Looking forward to your feedback!</p>
<hr/>**评论：**<br/><br/>shadowmint: <pre><p>I think the short answer is 'yes but...' you probably don't want to build one of those right now.</p>
<p>If you're keen, read this thread -> <a href="https://groups.google.com/forum/#!topic/golang-dev/Og1p9ECURBI" rel="nofollow">https://groups.google.com/forum/#!topic/golang-dev/Og1p9ECURBI</a></p>
<p>...and get involved with the working group.</p></pre>drmyersii: <pre><p>I really appreciate this! I've been looking for a group dedicated to solving this issue, so this might be exactly what I need!</p></pre>daniel_chatfield: <pre><p>As <a href="/u/shadowmint" rel="nofollow">/u/shadowmint</a> mentioned, there is a working group to try and solve this problem now.</p>
<p>There is no perfect solution but I'm looking forward to what the working group comes up with.</p>
<blockquote>
<p>One of the first things I'm going to be releasing is a service similar to gopkg.in, but it will actually allow you to specify any semantic version tag (v0.1.2, v0.2.1, v1.3.4, etc) for the repo endpoint</p>
</blockquote>
<p>What do you mean by this? Do you mean I can specify an explicit version number and I'll get that version? If so, that means every time a dependency is updated the version number has to be updated - that doesn't really add much on top of the existing tools for pinning to a particular revision. The whole point of semver is that the version number indicates compatibility with previous versions such that a dependency list doesn't need to be updated.</p>
<p>If instead, hitting <a href="https://foo.com/package@1.2.5" rel="nofollow">https://foo.com/package@1.2.5</a> gives you the highest semantically equivalent package then this is quite strange behaviour.</p>
<p>What if package <code>A</code> depends on package <code>B</code> and package <code>C</code>, B asks for <code>D@^1.0.0</code> and <code>C</code> asks for <code>D@1.2.0</code>, you now have to do version constraint resolution which is especially complex with Go as you don't know the dependencies of your dependencies until you've installed them (unlike other languages that have a file to explicitly list them).</p>
<p>It is for these reasons that I actually quite like the current state of affairs with Go:</p>
<ul>
<li>plain old <code>go get</code>ting works 99% of the time and has approximately zero complexity</li>
<li>it creates strong incentives to properly think about what you export from a package as breaking backwards compatibility is difficult</li>
<li><code>vendoring</code> means that if you have more complex requirements then these can be achieved by something external which can then stuff the result into the vendor directory.</li>
</ul></pre>drmyersii: <pre><p>I definitely see your reasoning (as well as some of the Go team/community), but what some think is a solution to dependency management is actually a hindrance. There are no guarantees in SemVer that allow someone to fully trust all packages >= 1.x, and this is why other package managers don't limit your specification. There are always bugs that make it through testing, and by not allowing you to lock your dependencies at a specific MAJOR.MINOR.PATCH version, you are opening yourself to the possibility of your code breaking due to someone else's release. I'm not saying you should have this functionality, because you should, but I am saying that you should be able to circumvent it if you so desire. <strong>This is one of the biggest reasons Go authors don't like to consume third-party packages.</strong></p>
<blockquote>
<p>What if package A depends on package B and package C, B asks for D@^1.0.0 and C asks for <a href="mailto:D@1.2.0" rel="nofollow">D@1.2.0</a></p>
</blockquote>
<p>This isn't exactly a new problem, and many other package managers attack it in many different ways. The idea of installing both versions isn't a bad one necessarily, but I wouldn't go the route of a nested dependency tree like NPM did.</p></pre>daniel_chatfield: <pre><p>Completely agree that the ability to pin to particular versions is useful ala <code>pip freeze</code>, <code>npm shrinkwrap</code>.</p>
<p>There are existing tools e.g. Godeps that allow you to do just that. </p>
<blockquote>
<p>The idea of installing both versions isn't a bad one necessarily</p>
</blockquote>
<p>In many cases this won't compile. If the dependency is one which doesn't leak into the exported interface then you're probably fine but if it does then you are probably going to have a bad time.</p>
<p>I guess what I'm trying to say is the current toolchain solves 70% of problem with 10% of the complexity. You can optionally plug in extra things that add significant complexity and solve another 20%.</p>
<p>The goal (as I see it) of the working group is to come up with a solution that with minimum added complexity can increase 70% to 85% and make it much easier for third party tools to fill the remaining 15% in a consistent manner.</p>
<p>Go is all about developer productivity. I work for a Bank - we currently have in excess of 100 Go services in production, the amount of productivity lost to this problem is tiny. We vendor everything and when something is updated we take a look and see whether we want to update it. This is much more of a problem in dynamic languages than in Go, where if it compiles you can be pretty sure it's working.</p></pre>wlll: <pre><p>Things I want:</p>
<ul>
<li>Able to specify version like Bundler can, so "<= 5" or "~> 0.1.2" (any patch level under 0.1)</li>
<li>Allow for local development of the lib. If I fork a lib to make changes I want to be able to use my fork.</li>
</ul>
<p>Basically Bundler in Ruby but for Go. Bundler works really well and I miss it in the Go world.</p></pre>drmyersii: <pre><p>Bundler is one of my main influences! :)</p></pre>wlll: <pre><p>I was using Goop, but I'd seems to be I maintained now. No support for current vendoring.</p></pre>TheMerovius: <pre><blockquote>
<p>no platform for publishing actual packages to</p>
</blockquote>
<p>I said it before, I say it again: The decentralized nature of import path based fetching is a <em>strength</em>, not a weakness.</p>
<blockquote>
<p>Does this sound like it would be useful?</p>
</blockquote>
<p>I'm sorry, but no, it doesn't. The last thing go needs is another island-solution "package-manager", which this is bound to become, promises or not. There are almost as many package managers as http routers. And they completely worthless if everyone uses a different one (and from what you said as properties, I can promise you that your's won't be the one to rule them all, there are too many people who disagree with the choices you made).</p>
<p>If you want to change the packaging landscape of go (and I still see no need for that, but that is just my opinion), participate in the debate around problems and solutions currently going on, because that might actually end up creating something useful to the majority.</p></pre>drmyersii: <pre><p>A somewhat harsh response to a not-so-harsh question/statement, but I expected as much. Do you truly believe that Go package management is fine where it currently sits? I fully intend to start interacting with the working group, and even they seem to be expressing issues very similar to what I'm saying. The nature of not being able to lock dependencies (without including them in the repo itself) and update them when you're ready and have tested the update is a step back in development. </p></pre>TheMerovius: <pre><blockquote>
<p>Do you truly believe that Go package management is fine where it currently sits?</p>
</blockquote>
<p>Mostly, yeah. I don't believe what we need is better package management, but if anything, better tooling around finding the API version we need. go get checks out git (or whatever) repositories, you can put them at any version that you want, there really is no deeper need to talk about "packaging". go repositories should just be repos. It is annoying if things stop building, but you can work around that pretty well, IMHO, by looking at the set of repos in your GOPATH and then having a tool that determines a set of commits that will make everything build just fine.</p>
<p>There is a valid concern about reproducible builds, but that, again, doesn't really need "packages", it just needs a description of the state of your GOPATH (and that doesn't need to be put into the repo, it's fine distributing it along with the ready-built binaries that you release and sign, for people to verify that their artifact is identical and you didn't cheat).</p>
<p>There is a valid concern about upstream repos becoming unavailable, but, again, the solution isn't "packaging" (lest alone a central repository of packages, which will make things <em>less</em> reliable, not more), but maintaining mirrors.</p>
<p>People complain about GOPATH making it impossible to fork packages, but that is simply untrue; you get a git-repository in your GOPATH, just add your own repo as a separate remote, commit there, push, make a PR, it's business as usual, as with every other language too. You can't have a long-term maintained fork under the same package name, without major import-path rewriting, but again, <em>that's a good thing</em>. If it's a different package, it <em>should</em> have a different name. Import paths as an identifier and location are a <em>great</em> solution, because you a) get automatic consensus over who owns a package, enforced by an external system (DNS) and b) everyone has complete control over their namespace (as opposed to a centralized repo, where things can just get taken offline on a whim, see the whole left-pad scenario for an example). So, in the end, any "central repository" will just end up implementing <em>the same</em> notion of more-or-less-url-based identifiers and at that point you simply get what I said above: A mirror.</p>
<p>I <em>do</em> see issues with go-package management (as opposed to "go package-management"), but the solution is not to clone the horrifically broken npm/gem/whatever.</p>
<blockquote>
<p>even they seem to be expressing issues very similar to what I'm saying.</p>
</blockquote>
<p>I didn't say my opinion was mainstream, but it <em>is</em> my opinion and you asked for that :)</p>
<blockquote>
<p>The nature of not being able to lock dependencies (without including them in the repo itself) and update them when you're ready and have tested the update is a step back in development.</p>
</blockquote>
<p>There never was any of my dependencies updated without my express action, so…
And yes, you shouldn't release untested code into your production environment, but that's an operational and release management concern, not a developmental. In particular, it's not something that upstream can or should solve; They won't know what you are running in production and how it looks like and it's unfeasible to expect them to accommodate that.</p></pre>drvd: <pre><p>There is no solution to a provable unsolvable problem.</p></pre>drmyersii: <pre><p>And what makes you say this? How can a problem that has been solved in so many other languages be unsolvable?</p></pre>TheMerovius: <pre><blockquote>
<p>How can a problem that has been solved in so many other languages be unsolvable?</p>
</blockquote>
<p>I (and I assume <a href="/u/drvd" rel="nofollow">/u/drvd</a>) disagree with your premise that it has been "solved in other languages". As an example, I am avoiding ruby software, because the constant breakages and the whole version fubar is just not bearable to deal with. And gem is usually touted as a good example of how it's done. And then you ask someone doing ruby and they'll be like "yeah, well, obviously you want to use rvm" and at that point it's just a table-flip.</p>
<p>[edit] also don't forget npm, which is the other thing touted as having "solved" that problem and look at the whole left-pad disaster for proof that it actually hasn't solved anything…</p></pre>drvd: <pre><p>E.g. conflicting diamond dependencies need manual resolution. And: Stuff like "requires at least 2.7.8 but not 2.9.1" is no guarantee that 2.9.7 won't break.
The problem is "solvable" only by handpicking a global set of version of all packages, thoroughly test testing them all in combination and declaring victory for now.</p></pre>neofreeman: <pre><p>Sounds useful to me if it fixes dependency tree and versioning issues. To put it short, I wish for a package manager as good as npm or gem/bundler</p></pre>mcvoid1: <pre><p>I don't. NPM drives me nuts. (Though it has been getting better. Flattening the node_modules structure and removing peer dependencies was a big plus.) The thing that (still) drives me nuts is the fact that NPM allows you to update versions without changing the version number. A number of javascript libraries (including react, which I hate them for) like to update the dependencies in a single version, so that even if you have the version numbers locked down, you still end up having a different product that will suddenly fail to build when you don't have access to the registry. Having a commit build one day and fail the next because of an update in NPM makes my blood boil.</p></pre>drmyersii: <pre><p>Glad to hear it!</p></pre>sixfootnix: <pre><p>Another look at it is this. With package managers like nix and guix you can fetch packages (source or compiled packages) with their cryptographic hashes. This means you have the exact version of a package you need, including all dependencies with their exact versions.</p>
<p>This calls for compiling with a recipe file outside of the language. Then you just say "directory.xyz is modulename1" in the recipe file.</p>
<p>It's just an idea but I think it's worth to let the OS package manager do the package management instead of the language especially with exact package managers like nix and guix.</p></pre>
