<section id="nice" data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="font-size: 16px; color: black; padding: 0 10px; line-height: 1.6; word-spacing: 0px; letter-spacing: 0px; word-break: break-word; word-wrap: break-word; text-align: left; font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, 'PingFang SC', Cambria, Cochin, Georgia, Times, 'Times New Roman', serif;"><h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; border-bottom: 2px solid rgb(239, 112, 96); font-size: 1.3em;"><span class="prefix" style="display: none;"></span><span class="content" style="display: inline-block; font-weight: bold; background: rgb(239, 112, 96); color: #ffffff; padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">背景</span><span class="suffix"></span><span style="display: inline-block; vertical-align: bottom; border-bottom: 36px solid #efebe9; border-right: 20px solid transparent;"> </span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">随着云原生技术的日益发展和普及,微服务架构下多语言(python,go,php....)多系统之间通过rpc来通信,一旦接口需要升级,协调 server 端有发布和升级,IDL 文件仓库的变更,client 端变更发布就变成了一个很麻烦的事情。我们需要一个方便透明的规范来协调各端,一个更加自动化的工具来协同。以主流的grpc为例来探讨下, 这个 proto 及代码到底放在哪里?应该如何做到自动化版本控制?</p>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125222844.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; border-bottom: 2px solid rgb(239, 112, 96); font-size: 1.3em;"><span class="prefix" style="display: none;"></span><span class="content" style="display: inline-block; font-weight: bold; background: rgb(239, 112, 96); color: #ffffff; padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">方案一:放在各自的代码仓库</span><span class="suffix"></span><span style="display: inline-block; vertical-align: bottom; border-bottom: 36px solid #efebe9; border-right: 20px solid transparent;"> </span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">如下图所示,直接将项目所依赖到的以及自己的所有 proto 文件和client代码都存放在 <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: rgb(239, 112, 96);">protobuf/</code> 目录下,不借助外部工具。</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;"><img src="https://cdn.hanyajun.com/20201125222907.png" alt style="display: block; margin: 0 auto; max-width: 100%;">
很明显该方案有以下优缺点。</p>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px;"><span class="prefix" style="display: none;"></span><span class="content">缺点:</span><span class="suffix" style="display: none;"></span></h3>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">1️⃣ 项目所有依赖的 proto 都存储在各自项目的代码仓库下,因此所有依赖 proto 都需要人工的向其它业务组 “要” 来,再放到 <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: rgb(239, 112, 96);">protobuf/</code> 目录下,人工介入极度麻烦。依赖多了拷贝会成为一种负担。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">2️⃣ 版本的控制和更新没有办法去维护,沟通成本太高。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">3️⃣ 代码的复用性比较差。</p>
</section></li></ul>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px;"><span class="prefix" style="display: none;"></span><span class="content">优点:</span><span class="suffix" style="display: none;"></span></h3>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">简单易用,每个项目的proto依赖一目了然。不需要看这看那。</section></li></ul>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; border-bottom: 2px solid rgb(239, 112, 96); font-size: 1.3em;"><span class="prefix" style="display: none;"></span><span class="content" style="display: inline-block; font-weight: bold; background: rgb(239, 112, 96); color: #ffffff; padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">方案二:独立proto仓库</span><span class="suffix"></span><span style="display: inline-block; vertical-align: bottom; border-bottom: 36px solid #efebe9; border-right: 20px solid transparent;"> </span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">每个项目都有自己的proto仓库。需要依赖的都从proto仓库拉(包括服务自己)。</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;"><img src="https://cdn.hanyajun.com/20201125223327.png" alt style="display: block; margin: 0 auto; max-width: 100%;">
不过这种方式也是有比较明显的缺点:</p>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px;"><span class="prefix" style="display: none;"></span><span class="content">缺点:</span><span class="suffix" style="display: none;"></span></h3>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">1️⃣ 各服务需要同时关注自己的服务本身还有服务的proto仓库,开发时需要先在proto仓库定义好proto和生成好client代码,所以在开发时需要在服务仓库和proto仓库切换。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">2️⃣ 如果服务依赖的太多,也有可能跨业务组,例如下图,如果跨组之后,可能需求找业务组负责人开多个proto仓库权限。还需要一个一个地去依赖。</p>
</section></li></ul>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125223356.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">3️⃣ 每次有新的服务都需要对应去申请一个proto仓库比较麻烦。</section></li></ul>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px;"><span class="prefix" style="display: none;"></span><span class="content">优点:</span><span class="suffix" style="display: none;"></span></h3>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">1️⃣ 各自服务有各自的proto仓库,方便版本维护和升级。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">2️⃣ 依赖时可以按需拉取。</p>
</section></li></ul>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; border-bottom: 2px solid rgb(239, 112, 96); font-size: 1.3em;"><span class="prefix" style="display: none;"></span><span class="content" style="display: inline-block; font-weight: bold; background: rgb(239, 112, 96); color: #ffffff; padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">方案三:集中仓库</span><span class="suffix"></span><span style="display: inline-block; vertical-align: bottom; border-bottom: 36px solid #efebe9; border-right: 20px solid transparent;"> </span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">按业务组维度来管理proto仓库,这样的话如果依赖某个业务组多个依赖时只需要拉取一个proto仓库即可。同时每次新起一个服务时,只需要在各自业务组的prooto仓库加上自己服务的,不需要单独去申请一个自己服务的proto仓库。</p>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125223755.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px;"><span class="prefix" style="display: none;"></span><span class="content">优点:</span><span class="suffix" style="display: none;"></span></h3>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">1️⃣ 依赖多个服务的proto时只需要依赖中央仓库即可。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">2️⃣ 新增服务时不需要单独去申请proto仓库,只需要在各自业务组的proto仓库添加即可。</p>
</section></li></ul>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px;"><span class="prefix" style="display: none;"></span><span class="content">缺点:</span><span class="suffix" style="display: none;"></span></h3>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">1️⃣ 各服务开发时还是需要关注两个仓库,需要切来切去。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">2️⃣ 每个中央仓库下不能独立的对各服务的proto进行版本管理。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">3️⃣ 可能只依赖某个svc的proto,而多引入了一些其他不必要的。</p>
</section></li></ul>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; border-bottom: 2px solid rgb(239, 112, 96); font-size: 1.3em;"><span class="prefix" style="display: none;"></span><span class="content" style="display: inline-block; font-weight: bold; background: rgb(239, 112, 96); color: #ffffff; padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">方案四:镜像仓库+git branch</span><span class="suffix"></span><span style="display: inline-block; vertical-align: bottom; border-bottom: 36px solid #efebe9; border-right: 20px solid transparent;"> </span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">为解决上述方案的一些痛点,综合各方案的优点我们在中央仓库的基础之上采取了镜像仓库结合git branch的方式。
<img src="https://cdn.hanyajun.com/20201125223821.png" alt style="display: block; margin: 0 auto; max-width: 100%;">
说明:</p>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">1️⃣ 各业务组的中央仓库的master的readme维护了各分支和各服务的对应关系,如下图。</section></li></ul>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125223902.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">2️⃣ 这样主要是为了新增新的svc时可以拿到下一个分支的分支名(由于go.mod的分支版本管理限制,只能以V2,V3这种命名),同时方便人们去辨识自己依赖的svc在哪个模块。common表示是所有svc proto的集合分支。每次新增一个服务时便会在cicd的过程中往这个reame添加说明。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">3️⃣ 中央仓库的分支的readme维护了各自自己的版本情况。</p>
</section></li></ul>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125224048.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">4️⃣ 使用时可以根据自己的需要去依赖对应的分支的版本即可。</p>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">5️⃣ python cleint版本和go的版本都是一一对应的。</p>
</section></li></ul>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125224114.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">6️⃣ 各svc只需要依赖一个.gitlab.yml文件即可实现。具体的gitlab cicd的job如下:</p>
<pre class="custom" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/point.png); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #282c34; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; display: -webkit-box; -webkit-overflow-scrolling: touch; font-size: 14px; word-wrap: break-word; padding: 2px 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: rgb(239, 112, 96); padding-top: 15px; background: #282c34; border-radius: 5px;"><br><span class="hljs-string" style="color: #98c379; line-height: 26px;">.push_tmpl:</span> <span class="hljs-meta" style="color: #61aeee; line-height: 26px;">&push_proto</span><br> <span class="hljs-attr" style="color: #d19a66; line-height: 26px;">script:</span><br> <span class="hljs-bullet" style="color: #61aeee; line-height: 26px;">-</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">"push test"</span><br> <span class="hljs-bullet" style="color: #61aeee; line-height: 26px;">-</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$CI_PROJECT_NAME</span><br> <span class="hljs-bullet" style="color: #61aeee; line-height: 26px;">-</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">|<br> userMail=$GITLAB_USER_EMAIL<br> git config --global user.email "$GITLAB_USER_EMAIL"<br> userName=${userMail%@*}<br> echo "$userMail"<br> git config --global user.name "$userName"<br> git clone -v https://xxxxx/proto-center.git<br> cd proto-center<br> if [ `grep -c $CI_PROJECT_NAME README.md` -eq '0' ];then<br> echo "- $CI_PROJECT_NAME-->v$(sed -n '$p' README.md | awk -F "-->v" '{print $2+1}' | head)" >>README.md<br> export branchName=$(sed -n '$p' README.md | awk -F "-->" '{print $2}')<br> echo $branchName<br> git add . && git commit -m "add $CI_PROJECT_NAME "<br> git push https://xxxxxxxx/proto-center.git master<br> git checkout -b $branchName<br> mkdir -p $CI_PROJECT_NAME && cp -r ../protobuf/* $CI_PROJECT_NAME/<br> go mod init xxxxxxxx/proto-center/$branchName<br> go mod tidy<br> git add .<br> git commit -m "add $CI_PROJECT_NAME proto"<br> git push --set-upstream https:/xxxxxxxx/proto-center.git $branchName<br> git checkout v2<br> mkdir -p $CI_PROJECT_NAME && cp -r ../protobuf/* $CI_PROJECT_NAME/<br> git add .<br> git commit -m "add $CI_PROJECT_NAME proto"<br> git push https://xxxxxxxx/proto-center.git v2<br> else<br> export branchName=$(grep $CI_PROJECT_NAME README.md | awk -F "-->" '{print $2}' | head)<br> git checkout $branchName<br> rm -rf $CI_PROJECT_NAME/<br> mkdir -p $CI_PROJECT_NAME<br> cp -r ../protobuf/* $CI_PROJECT_NAME/ && git add .<br> git commit -m "update $CI_PROJECT_NAME proto"<br> git push https://xxxxxxxx/proto-center.git $branchName;<br> git checkout v2<br> rm -rf $CI_PROJECT_NAME/<br> mkdir -p $CI_PROJECT_NAME<br> cp -r ../protobuf/* $CI_PROJECT_NAME/ && git add .<br> git commit -m "update $CI_PROJECT_NAME proto"<br> git push https://xxxxxxxx/proto-center.git v2;<br> fi<br></span></code></pre>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">7️⃣ 中央仓库的gitlab的cicd的job如下:</p>
<pre class="custom" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/point.png); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #282c34; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; display: -webkit-box; -webkit-overflow-scrolling: touch; font-size: 14px; word-wrap: break-word; padding: 2px 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: rgb(239, 112, 96); padding-top: 15px; background: #282c34; border-radius: 5px;"><span class="hljs-string" style="color: #98c379; line-height: 26px;">.buld_tmpl:</span> <span class="hljs-meta" style="color: #61aeee; line-height: 26px;">&tag_proto</span><br><br> <span class="hljs-attr" style="color: #d19a66; line-height: 26px;">script:</span><br> <span class="hljs-bullet" style="color: #61aeee; line-height: 26px;">-</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">"tag proto "</span><br> <span class="hljs-bullet" style="color: #61aeee; line-height: 26px;">-</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">|<br> userMail=$GITLAB_USER_EMAIL<br> userName=${userMail%@*}<br> echo "$userMail"<br> git config --global user.email $userMail<br> git config --global user.name $userName<br> branch=$CI_BUILD_REF_NAME<br> isnottag="false"<br> git describe --tag || isnottag="true"<br> git clone -b $branch -v https://xxxxx/proto-center.git<br> cd proto-center<br> userMail=$( git log --pretty=format:%ae ${CI_COMMIT_SHA} -1)<br> userName=${userMail%@*}<br> echo "$userMail"<br> git config --global user.email $userMail<br> git config --global user.name $userName<br> version=""<br> if [ $isnottag = "true" ];then<br> echo $isnottag<br> version=$branch.0.1<br> echo $version<br> git tag $version<br> git push https://xxxxx/proto-center.git --tags<br> echo "- $version" >> README.md<br> git add .<br> git commit -m "add $version"<br> git push https://xxxxx/proto-center.git $branch<br> else<br> echo $isnottag<br> version_ref=$(git describe --tags | awk -F "-" '{print $1}' | head)<br> echo $version_ref<br> version=`echo $version_ref | awk -F. -v OFS=. 'NF==1{print ++$NF}; NF>1{if(length($NF+1)>length($NF))$(NF-1)++; $NF=sprintf("%0*d", length($NF), ($NF+1)%(10^length($NF))); print}'`<br> echo $version<br> git tag $version<br> git push https://xxxxx/proto-center.git --tags<br> echo "- $version" >> README.md<br> git add .<br> git commit -m "add $version"<br> git push https://xxxxx/proto-center.git $branch<br> fi<br> mkdir -p python<br> cp -r **/*.proto python<br> pkgName=$( grep "\\-->" README.md | tail -1 | awk -F "-->v" '{print $1}' | awk -F " " '{print $2}')<br> pkgName=${pkgName//-/_}<br> echo $pkgName<br> export pip_pkg_name=$pkgName<br> echo $version<br> export pip_tag_name=$version<br> pip install grpcio-tools==1.4.0<br> pip install protobuf==3.3.0<br> echo "#!/bin/env python<br> # -*- encoding=utf8 -*-<br> import os<br> from setuptools import (setup, find_packages)<br> version = os.getenv('pip_tag_name')<br> name = os.getenv('pip_pkg_name')<br> setup(<br> name=name,<br> version=version,<br> description='LLS grpc protocol',<br> packages=find_packages(exclude=[]),<br> include_package_data=True,<br> author='LLS DEV Team',<br> author_email='',<br> package_data={'': ['*.*']},<br> install_requires=[<br> 'grpcio==1.18.0',<br> 'protobuf==3.3.0'<br> ],<br> zip_safe=False,<br> classifiers=[<br> 'Programming Language :: python :: 2.7',<br> ],<br> )" > python/setup.py<br> echo "<br> [distutils]<br> index-servers = internal<br></span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">[internal]</span><br> <span class="hljs-attr" style="color: #d19a66; line-height: 26px;">repository:</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">https://xxxxx.com/</span><br> <span class="hljs-attr" style="color: #d19a66; line-height: 26px;">username:</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">xxxxx</span><br> <span class="hljs-attr" style="color: #d19a66; line-height: 26px;">password:</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">xxxxx</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">" > ~/.pypirc<br> mkdir -p python/$pkgName<br> echo "</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">" >> python/$pkgName/__init__.py<br> python -m grpc_tools.protoc -I python/ --python_out=python/$pkgName --grpc_python_out=python/$pkgName/ python/*.proto<br> cd python<br> ls<br> python setup.py bdist_wheel upload -r internal<br> cd ..<br> mkdir -p python3<br> cp -r -n **/*.proto python3<br> echo "</span><span class="hljs-comment" style="color: #5c6370; font-style: italic; line-height: 26px;">#!/bin/env python</span><br> <span class="hljs-comment" style="color: #5c6370; font-style: italic; line-height: 26px;"># -*- encoding=utf8 -*-</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">import</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">os</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">from</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">setuptools</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">import</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">(setup,</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">find_packages)</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">version</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">=</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">os.getenv('pip_tag_name')</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">name</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">=</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">os.getenv('pip_pkg_name')</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">setup(</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">name=name,</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">version=version,</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">description='LLS</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">grpc</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">protocol',</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">packages=find_packages(exclude=[]),</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">include_package_data=True,</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">author='LLS</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">DEV</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">Team',</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">author_email='',</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">package_data={'':</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">['*.*']},</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">install_requires=[</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'grpcio==1.18.0'</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">,</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'protobuf==3.12.4'</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">],</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">zip_safe=False,</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">classifiers=[</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'Programming Language :: python :: 3.7'</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">,</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">],</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">)"</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">></span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3/setup.py</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-m</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">pip</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">install</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">grpcio-tools==1.4.0</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-m</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">pip</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">install</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">protobuf</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">mkdir</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-p</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3/$pkgName</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">""</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">>></span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3/$pkgName/__init__.py</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-m</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">grpc_tools.protoc</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-I</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3/</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">--python_out=python3/$pkgName</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">--grpc_python_out=python3/$pkgName/</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3/*.proto</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">cd</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3/$pkgName</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">ls</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">pb_files=`ls</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">|</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">grep</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-v</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'__init__'</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">|</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">grep</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-v</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'grpc.py'</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">`</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$pb_files</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">need_replace_strs=()</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">for</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">pb_file</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">in</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">${pb_files[@]}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">do</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">prefix=${pb_file/.py/}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">after_handle_package_name=${prefix//_/__}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">need_replace_str="import</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$prefix</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">as</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$after_handle_package_name"</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$need_replace_str</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">need_replace_strs[${#need_replace_strs[@]}]="$need_replace_str"</span><br> <span class="hljs-comment" style="color: #5c6370; font-style: italic; line-height: 26px;">#echo ${need_replace_strs[0]}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">done</span><br> <br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">need_replace_str_num=${#need_replace_strs[@]}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">all_files=`ls</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">|</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">grep</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-v</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'__init__'</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">`</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$all_files</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">for</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">file</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">in</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">${all_files[@]}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">do</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">for</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">((i=0;i<$need_replace_str_num;i++))</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">do</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">need_replace_str=${need_replace_strs[${i}]}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$need_replace_str</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">sed</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-i</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">"s/^$need_replace_str$/from . $need_replace_str/"</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$file</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">done</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">done</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">"finished"</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">cd</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">..</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">python3</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">setup.py</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">bdist_wheel</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">upload</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">-r</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">internal</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">echo</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">${CI_COMMIT_SHA}</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">noticeMail=$userMail</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">content="\nversion:</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$version\nproject:</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">$pkgName\ncommit:\nhttps://xxxxx/proto-center/commit/${CI_COMMIT_SHA}\n"</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">curl</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">--location</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">--request</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">POST</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'https://xxxxx/webhookurl'</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">\</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">--header</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'Content-Type: application/json'</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">\</span><br> <span class="hljs-string" style="color: #98c379; line-height: 26px;">--data-raw</span> <span class="hljs-string" style="color: #98c379; line-height: 26px;">'{<br> "msg": {<br> "content":"'</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">"${content}"</span><span class="hljs-string" style="color: #98c379; line-height: 26px;">'"<br> }'</span><br></code></pre>
</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;"><p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">8️⃣ 构建完通知,在中央仓库的job里面可以添加钉钉,微信,slack这种类似的机器人🤖通知方便我们知道proto cicd流程的情况,可以带上我们的commit信息以及对应的版本。</p>
</section></li></ul>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://cdn.hanyajun.com/20201125224142.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; border-bottom: 2px solid rgb(239, 112, 96); font-size: 1.3em;"><span class="prefix" style="display: none;"></span><span class="content" style="display: inline-block; font-weight: bold; background: rgb(239, 112, 96); color: #ffffff; padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">总结</span><span class="suffix"></span><span style="display: inline-block; vertical-align: bottom; border-bottom: 36px solid #efebe9; border-right: 20px solid transparent;"> </span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black;">总的来说,还有很多方案,每种方案都各有利弊,适合业务系统架构和公司组织机构的才是最好的方案,不过尽量通过工具来减少人肉的过程,提高效率。</p>
<blockquote class="multiquote-1" data-tool="mdnice编辑器" style="border: none; display: block; font-size: 0.9em; overflow: auto; overflow-scrolling: touch; border-left: 3px solid rgba(0, 0, 0, 0.4); color: #6a737d; padding-top: 10px; padding-bottom: 10px; padding-left: 20px; padding-right: 10px; margin-bottom: 20px; margin-top: 20px; border-left-color: rgb(239, 112, 96); background: #fff9f9;">
<p style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0px; color: black; line-height: 26px;">参考链接</p>
</blockquote>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">https://eddycjy.com/posts/where-is-proto</section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">https://segmentfault.com/a/1190000022532645</section></li></ul>
</section>
有疑问加站长微信联系(非本文作者))