gitlab bash

Few things are more satisfying to me than one elegant line of Bash that automates hours of tedious work.

沒有什么比讓Bash自動完成數小時繁瑣工作的Bash優雅系列令我滿意的了。

As part of some recent explorations into automatically re-creating my laptop with Bash scripts (post to come!), I wanted to find a way to easily clone my GitHub-hosted repositories to a new machine. After a bit of digging around, I wrote a one-liner that did just that.

作為最近使用Bash腳本自動重新創建筆記本電腦的探索的一部分，我想找到一種輕松地將GitHub托管的存儲庫克隆到新計算機上的方法。經過一番摸索之后，我寫了一篇這樣的單線紙。

Then, in the spirit of not putting all our eggs in the same basket, I wrote another one-liner to automatically create and push to GitLab-hosted backups as well. Here they are.

然后，本著不把所有雞蛋都放在同一籃子的精神，我寫了另一種單行代碼來自動創建并推送到GitLab托管的備份。他們來了。

一個Bash一線式克隆您所有的GitHub存儲庫 (A Bash one-liner to clone all your GitHub repositories)

Caveat: you’ll need a list of the GitHub repositories you want to clone. The good thing about that is it gives you full agency to choose just the repositories you want on your machine, instead of going in whole-hog.

注意：您將需要要克隆的GitHub存儲庫的列表。這樣做的好處是，它使您可以完全選擇僅要在計算機上存儲的存儲庫，而不必花很多精力。

You can easily clone GitHub repositories without entering your password each time by using HTTPS with your 15-minute cached credentials or, my preferred method, by connecting to GitHub with SSH. For brevity I’ll assume we’re going with the latter, and our SSH keys are set up.

您可以輕松地克隆GitHub存儲庫，而無需每次都輸入HTTPS并使用15分鐘的緩存憑據，或者，我首選的方法是使用SSH連接到GitHub，而無需每次輸入密碼。為簡便起見，我假設我們要使用后者，并且我們已經設置了SSH密鑰。

Given a list of GitHub URLs in the file gh-repos.txt, like this:

在文件gh-repos.txt中給出GitHub URL的列表，如下所示：

git@github.com:username/first-repository.git
git@github.com:username/second-repository.git
git@github.com:username/third-repository.git

We run:

我們跑：

xargs -n1 git clone < gh-repos.txt

This clones all the repositories on the list into the current folder. This same one-liner works for GitLab as well, if you substitute the appropriate URLs.

這會將列表上的所有存儲庫克隆到當前文件夾中。如果替換適當的URL，則同一行也適用于GitLab。

這里發生了什么？ (What’s going on here?)

There are two halves to this one-liner: the input, counterintuitively on the right side, and the part that makes stuff happen, on the left. We could make the order of these parts more intuitive (maybe?) by writing the same command like this:

單行代碼有兩半：輸入(在直覺上相反)在右側，而使事情發生的部分在左側。通過編寫如下相同的命令，我們可以使這些部分的順序更直觀(也許？)：

<gh-repos.txt xargs -n1 git clone

To run a command for each line of our input, gh-repos.txt, we use xargs -n1. The tool xargs reads items from input and executes any commands it finds (it will echo if it doesn’t find any). By default, it assumes that items are separated by spaces; new lines also works and makes our list easier to read. The flag -n1tells xargs to use 1 argument, or in our case, one line, per command. We build our command with git clone, which xargs then executes for each line. Ta-da.

要對輸入的每一行gh-repos.txt運行命令，我們使用xargs -n1 。 xargs工具從輸入中讀取項目并執行找到的所有命令(如果找不到則echo )。默認情況下，它假定項目之間用空格隔開；新行也可以使我們的列表更易于閱讀。標志-n1告訴xargs每個命令使用1參數，在本例中為1行。我們使用git clone構建命令，然后xargs將針對每一行執行。 -

Bash一線式工具，可在GitLab上創建并推送許多存儲庫 (A Bash one-liner to create and push many repositories on GitLab)

GitLab, unlike GitHub, lets us do this nifty thing where we don’t have to use the website to make a new repository first. We can create a new GitLab repository from our terminal. The newly created repository defaults to being set as Private, so if we want to make it Public on GitLab, we’ll have to do that manually later.

與GitHub不同，GitLab讓我們可以做這件漂亮的事情，而不必先使用網站創建新的存儲庫。我們可以從終端創建一個新的GitLab存儲庫。新創建的存儲庫默認設置為“私有”，因此，如果要在GitLab上將其設置為“公共”，則稍后必須手動進行。

The GitLab docs tell us to push to create a new project using git push --set-upstream, but I don’t find this to be very convenient for using GitLab as a backup. As I work with my repositories in the future, I’d like to run one command that pushes to both GitHub and GitLab without additional effort on my part.

GitLab文檔告訴我們使用git push --set-upstream來推動創建一個新項目，但是我發現使用GitLab作為備份不是很方便。將來在使用存儲庫時，我想運行一個命令同時推送到GitHub 和 GitLab，而無需我付出額外的努力。

To make this Bash one-liner work, we’ll also need a list of repository URLs for GitLab (ones that don’t exist yet). We can easily do this by copying our GitHub repository list, opening it up with Vim, and doing a search-and-replace:

為了使此Bash單線工作，我們還需要GitLab的存儲庫URL列表(尚不存在的URL)。我們可以輕松地做到這一點，方法是復制我們的GitHub存儲庫列表，使用Vim打開它，然后進行搜索和替換：

cp gh-repos.txt gl-repos.txt
vim gl-repos.txt
:%s/\<github\>/gitlab/g
:wq

This produces gl-repos.txt, which looks like:

這將產生gl-repos.txt ，看起來像：

git@gitlab.com:username/first-repository.git
git@gitlab.com:username/second-repository.git
git@gitlab.com:username/third-repository.git

We can create these repositories on GitLab, add the URLs as remotes, and push our code to the new repositories by running:

我們可以在GitLab上創建這些存儲庫，將URL添加為遠程存儲，并通過運行以下命令將代碼推送到新的存儲庫：

awk -F'\/|(\.git)' '{system("cd ~/FULL/PATH/" $2 " && git remote set-url origin --add " $0 " && git push")}' gl-repos.txt

Hang tight and I’ll explain it; for now, take note that ~/FULL/PATH/ should be the full path to the directory containing our GitHub repositories.

請稍等，我會解釋。現在，請注意~/FULL/PATH/應該是包含我們的GitHub存儲庫的目錄的完整路徑。

We do have to make note of a couple assumptions:

我們必須注意以下兩個假設：

The name of the directory on your local machine that contains the repository is the same as the name of the repository in the URL (this will be the case if it was cloned with the one-liner above);
包含資源庫的本地計算機上目錄的名稱與URL中資源庫的名稱相同(如果是使用上述單行代碼克隆的，則為這種情況)；
Each repository is currently checked out to the branch you want pushed, ie. master.
當前每個存儲庫都簽出到要推送的分支，即。 master 。

The one-liner could be expanded to handle these assumptions, but it is the humble opinion of the author that at that point, we really ought to be writing a Bash script.

單行代碼可以擴展以處理這些假設，但是作者的拙見認為，在那時，我們確實應該編寫Bash腳本。

這里發生了什么？ (What’s going on here?)

Our Bash one-liner uses each line (or URL) in the gl-repos.txt file as input. With awk, it splits off the name of the directory containing the repository on our local machine, and uses these pieces of information to build our larger command. If we were to print the output of awk, we’d see:

我們的Bash單行使用gl-repos.txt文件中的每一行(或URL)作為輸入。使用awk ，它拆分出包含本地計算機上存儲庫的目錄名稱，并使用這些信息來構建更大的命令。如果要print awk的輸出， awk看到：

cd ~/FULL/PATH/first-repository && git remote set-url origin --add git@gitlab.com:username/first-repository.git && git push
cd ~/FULL/PATH/second-repository && git remote set-url origin --add git@gitlab.com:username/second-repository.git && git push
cd ~/FULL/PATH/third-repository && git remote set-url origin --add git@gitlab.com:username/third-repository.git && git push

Let’s look at how we build this command.

讓我們看看我們如何構建此命令。

用`awk`分割字符串 (Splitting strings with `awk`)

The tool awk can split input based on field separators. The default separator is a whitespace character, but we can change this by passing the -F flag. Besides single characters, we can also use a regular expression field separator. Since our repository URLs have a set format, we can grab the repository names by asking for the substring between the slash character / and the end of the URL, .git.

工具awk可以基于字段分隔符拆分輸入。默認的分隔符是空格字符，但是我們可以通過傳遞-F標志來更改它。除了單個字符，我們還可以使用正則表達式字段分隔符。由于我們的存儲庫URL具有固定的格式，因此我們可以通過請求斜杠/和URL末尾.git之間的子字符串來獲取存儲庫名稱。

One way to accomplish this is with our regex \/|(\.git):

實現此目的的一種方法是使用我們的正則表達式\/|(\.git) ：

\/ is an escaped / character;
\/是轉義的/字符；
| means “or”, telling awk to match either expression;
| 表示“或”，告訴awk匹配任一表達式；
(\.git) is the capture group at the end of our URL that matches “.git”, with an escaped . character. This is a bit of a cheat, as “.git” isn’t strictly splitting anything (there’s nothing on the other side) but it’s an easy way for us to take this bit off.
(\.git)是URL末尾的捕獲組，與“ .git”匹配，并帶有一個轉義的. 字符。這有點作弊，因為“ .git”并沒有嚴格分割任何內容(另一面沒有任何內容)，但這是我們輕松實現這一點的簡便方法。

Once we’ve told awk where to split, we can grab the right substring with the field operator. We refer to our fields with a $ character, then by the field’s column number. In our example, we want the second field, $2. Here’s what all the substrings look like:

告訴awk拆分位置后，我們可以使用field運算符來獲取正確的子字符串。我們用$字符引用字段，然后用字段的列號引用。在我們的示例中，我們需要第二個字段$2 。這是所有子字符串的樣子：

1: git@gitlab.com:username
2: first-repository

To use the whole string, or in our case, the whole URL, we use the field operator $0. To write the command, we just substitute the field operators for the repository name and URL. Running this with print as we’re building it can help to make sure we’ve got all the spaces right.

要使用整個字符串，或者在我們的示例中，使用整個URL，我們使用字段運算符$0 。要編寫命令，我們只需將字段運算符替換為存儲庫名稱和URL。在構建它時使用print運行它可以幫助確保所有空間都正確。

awk -F'\/|(\.git)' '{print "cd ~/FULL/PATH/" $2 " && git remote set-url origin --add " $0 " && git push"}' gl-repos.txt

運行命令 (Running the command)

We build our command inside the parenthesis of system(). By using this as the output of awk, each command will run as soon as it is built and output. The system() function creates a child process that executes our command, then returns once the command is completed. In plain English, this lets us perform the Git commands on each repository, one-by-one, without breaking from our main process in which awk is doing things with our input file. Here’s our final command again, all put together.

我們在system()括號內構建命令。通過將其用作awk的輸出，每條命令在生成并輸出后將立即運行。 system()函數創建一個執行我們的命令的子進程，然后在命令完成后返回。用簡單的英語來說，這使我們可以在每個存儲庫上一對一地執行Git命令，而不會破壞awk使用輸入文件執行操作的主要過程。這是我們的最終命令，所有命令都放在一起了。

awk -F'\/|(\.git)' '{system("cd ~/FULL/PATH/" $2 " && git remote set-url origin --add " $0 " && git push")}' gl-repos.txt

使用我們的備份 (Using our backups)

By adding the GitLab URLs as remotes, we’ve simplified the process of pushing to both externally hosted repositories. If we run git remote -v in one of our repository directories, we’ll see:

通過將GitLab URL添加為遠程站點，我們簡化了推送到兩個外部托管存儲庫的過程。如果在其中一個存儲庫目錄中運行git remote -v ，我們將看到：

origin  git@github.com:username/first-repository.git (fetch)
origin  git@github.com:username/first-repository.git (push)
origin  git@gitlab.com:username/first-repository.git (push)

Now, simply running git push without arguments will push the current branch to both remote repositories.

現在，簡單地運行不帶參數的git push會將當前分支推送到兩個遠程存儲庫。

We should also note that git pull will generally only try to pull from the remote repository you originally cloned from (the URL marked (fetch) in our example above). Pulling from multiple Git repositories at the same time is possible, but complicated, and beyond the scope of this post. Here’s an explanation of pushing and pulling to multiple remotes to help get you started, if you’re curious. The Git documentation on remotes may also be helpful.

我們還應該注意， git pull通常只會嘗試從最初克隆的遠程存儲庫中(fetch)在上面的示例中標記為(fetch)的URL)。可以同時從多個Git存儲庫中拉出，但很復雜，超出了本文的范圍。如果您有好奇心，這是推和拉到多個遙控器以幫助您入門的說明。遙控器上的Git文檔也可能會有所幫助。

詳細闡述Bash單線的簡潔性 (To elaborate on the succinctness of Bash one-liners)

Bash one-liners, when understood, can be fun and handy shortcuts. At the very least, being aware of tools like xargs and awk can help to automate and alleviate a lot of tediousness in our work. However, there are some downsides.

如果了解Bash一線，可以說是有趣而便捷的捷徑。至少，了解xargs和awk類的工具可以幫助自動化和減輕我們工作中的繁瑣工作。但是，還有一些缺點。

In terms of an easy-to-understand, maintainable, and approachable tool, Bash one-liners suck. They’re usually more complicated to write than a Bash script using if or while loops, and certainly more complicated to read. It’s likely that when we write them, we’ll miss a single quote or closing parenthesis somewhere; and as I hope this post demonstrates, they can take quite a bit of explaining, too. So why use them?

就易于理解，易于維護和易于使用的工具而言，Bash一線難熬。與使用if或while循環的Bash腳本相比，編寫它們通常更復雜，并且讀取起來當然更復雜。當我們編寫它們時，可能會在某處遺漏單引號或右括號。正如我希望這篇文章所演示的那樣，他們也可以做很多解釋。那為什么要使用它們呢？

Imagine reading a recipe for baking a cake, step by step. You understand the methods and ingredients, and gather your supplies. Then, as you think about it, you begin to realize that if you just throw all the ingredients at the oven in precisely the right order, a cake will instantly materialize. You try it, and it works!

想象一下，逐步閱讀烘焙蛋糕的食譜。您了解方法和成分，并收集物品。然后，當您考慮它時，您開始意識到，如果只是以正確的順序將所有配料扔進烤箱，蛋糕就會立即變質。您嘗試一下，就可以了！

That would be pretty satisfying, wouldn’t it?

那會很令人滿意，不是嗎？