gitlab批量下载projects

• • • 参考文档
    • API
    • • 获取所有组
      • 获取所有项目
      • 根据项目获取组ID
      • 根据组查询项目
    • 批量下载projects使用说明
    • 批量获取项目名称
    • 进入项目中批量提交分支
    • 其他方法 python 方式

https://docs.gitlab.com/ee/api/projects.html#list-all-projects

curl --location --request GET 'https://git.rivtower.com/api/v4/namespaces' 
--header 'PRIVATE-TOKEN: EyQ65KR2i93dxbFPuw1M' 
--header 'Cookie: _gitlab_session=0876d79f9ff5b4a1d62ffd8836a22258'

curl --location --request GET 'https://git.rivtower.com/api/v4/projects?simple=true' 
--header 'PRIVATE-TOKEN: EyQ65KR2i93dxbFPuw1M' 
--header 'Cookie: _gitlab_session=0876d79f9ff5b4a1d62ffd8836a22258'

curl --location --request GET 'https://git.rivtower.com/api/v4/projects/350/groups' 
--header 'PRIVATE-TOKEN: EyQ65KR2i93dxbFPuw1M' 
--header 'Cookie: _gitlab_session=0876d79f9ff5b4a1d62ffd8836a22258'

curl --location --request GET 'https://git.rivtower.com/api/v4/groups/106' 
--header 'PRIVATE-TOKEN: EyQ65KR2i93dxbFPuw1M'

下载某个组下所有的项目（因为还没找到怎么去获取组中项目的size，所以使用预估的方式去试探）

#!/bin/bash
groupID=$1
projectSize=$2
for((i = 0; i < projectSize; i++)); do
echo $i
echo 'https://git.rivtower.com/api/v4/groups/'${groupID}
projectURL=`curl --location --request GET 'https://git.rivtower.com/api/v4/groups/'${groupID} --header 'PRIVATE-TOKEN: EyQ65KR2i93dxbFPuw1M' --header 'Cookie: _gitlab_session=0876d79f9ff5b4a1d62ffd8836a22258' | jq -r '.projects['$i'].http_url_to_repo'` 
echo "projectURL: " ${projectURL}
git clone ${projectURL}
done

运行说明：
第一个参数表示groupID
第二个参数表示project size

./getGitlabProjects.sh  106 2

其中groupID 的获取方式通过界面，获取到 Project ID

#!/bin/bash
test -e gitlabProjectName.txt && rm -rf gitlabProjectName.txt
groupID=$1
projectSize=$2
for((i = 0; i < projectSize; i++)); do
echo $i
echo 'https://git.rivtower.com/api/v4/groups/'${groupID}
name=`curl --location --request GET 'https://git.rivtower.com/api/v4/groups/'${groupID} --header 'PRIVATE-TOKEN: EyQ65KR2i93dxbFPuw1M' --header 'Cookie: _gitlab_session=0876d79f9ff5b4a1d62ffd8836a22258' | jq -r '.projects['$i'].name'`
echo ${name}  >> gitlabProjectName.txt
done

./getGitlabProjectsName.sh  106 2

#!/bin/bash
cat gitlabProjectName.txt | while read line
do
echo ${line}
cd  ${line}
time=`date "+%Y%m%d%H%M%S"`
branchName=sonar_${time}
echo ${branchName}
git checkout -b ${branchName}
git push --set-upstream origin ${branchName}
cd ../
done

./readName.sh 

# -*- coding: UTF-8 -*-

# 在Python3.0测试通过
# 需要在gitlab里面新建一个AccessToken填入gitlabToken

import sys
if sys.version_info < (3, 0):
    import urllib
else:
    from urllib.request import urlopen

import json
import subprocess, shlex
import time
import os

gitlabAddr  = 'git.rivtower.com'         #git的地址
gitlabToken = 'T6XJ8eKf75T6bkhzvGVb'    #gitlab的token，在gitlab的设置里有生产临时token

for index in range(10):
    url     = "http://%s/api/v4/projects?private_token=%s&per_page=100&page=%d&order_by=name" % (gitlabAddr, gitlabToken, index)
    print(url)

    if sys.version_info < (3, 0):
        allProjects     = urllib.urlopen(url)
    else:
        allProjects     = urlopen(url)

    allProjectsDict = json.loads(allProjects.read().decode(encoding='UTF-8'))
    if len(allProjectsDict) == 0:
        break
    for thisProject in allProjectsDict: 
        try:
            thisProjectURL  = thisProject['http_url_to_repo']
            thisProjectPath = thisProject['path_with_namespace']
            print(thisProjectURL + ' ' + thisProjectPath)

            if os.path.exists(thisProjectPath):
                command     = shlex.split('git -C "%s" pull' % (thisProjectPath))
            else:
                command     = shlex.split('git clone %s %s' % (thisProjectURL, thisProjectPath))

            resultCode  = subprocess.Popen(command)
            time.sleep(0.5)
        except Exception as e:
            print("Error on %s: %s" % (thisProjectURL, e.strerror))