如何将Secrets扫描加入到GitLab Pipeline

love374

发布于 2023-7-20 11:32

浏览

0收藏

概述

如果要管理应用程序开发管道以交付软件，则需要实施一项关键安全控制。您需要确保您的机密受到保护。简单地说，机密是您需要保护的凭据，因为该凭据具有特权功能。例如：

用户名和密码组合
接口密钥
JSON 网络令牌
私钥

开发人员经常意外地将这些凭据保存到存储库中。将这些凭据或机密添加到存储库后，清理可能会很痛苦，并且需要您更改凭据。更好的长期方法是使用“提交前检查”来避免存储机密。该主题将在另一篇文章中介绍。本博客重点介绍如何扫描现有存储库以查找 Gitlab 管道中存储的机密。

识别或创建存储库

如果您想继续操作，可以单击此链接获取公共仓库。https://gitlab.com/cmg_public/secrets_scanning_test

第一步是确定目标存储库。如果您正在入门但没有，只需在 Gitlab 中创建一个新项目即可。从菜单中单击“项目>创建新项目>导入上面项目地址”。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

扫描规则配置文件
文件路径：.gitlab/secret-detection-ruleset.toml

[secrets]
description = 'CMG DevSecOps Crash Course, secrets custom rules configuration'

[[secrets.passthrough]]
type = "raw"
target = "gitleaks.toml"
value = """\
title = "gitleaks config"
# Add regexes to the regex table
[[rules]]
description = "Test for Raw Custom Rulesets"
regex = '''Custom Raw Ruleset T[est]{3}'''

[[rules]]
description = "CMG Custom Generic Password"
regex = '''(?i)(password|passw)'''
tags = ["key", "Custom Password", "generic"]
"""1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.

您会注意到我添加了一些自定义规则。这些将用于标识默认情况下未启用的某些类型的机密。您可以根据需要创建任意数量的规则，以查找通常出现在开发人员代码中的模式。
"CMG Generic JWT" 此正则表达式（REGEX）查找存储在文件中的 JSON Web Token。诸如“JWT”、“jwt_token”、“secret”、“bearer”等。正则表达式不区分大小写。

[[rules]]
description = "CMG Generic JWT"
regex = '''(?i)(jwt|jwt_token|secret|bearer)'''
tags = ["key", "JWT", "generic"]1.
2.
3.
4.

"CMG Custom Generic Password" 此正则表达式查找存储在文件中的单词“password”的常见派生词。此正则表达式也不区分大小写。

[[rules]]
description = "CMG Custom Generic Password"
regex = '''(?i)(password|passw)'''
tags = ["key", "Custom Password", "generic"]
"""1.
2.
3.
4.
5.

Pipeline配置文件

此模板将用于根据默认设置和任何自定义正则表达式字符串（如我们在前面的步骤中添加的字符串）扫描机密。

# This file is a template, and might need editing before it works on your project.
# This template moved to Jobs/Secret-Detection.gitlab-ci.yml in GitLab 14.0
# Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/292977
include:
  template: Jobs/Secret-Detection.gitlab-ci.yml1.
2.
3.
4.
5.

此时，我们已准备好运行管道;但是，我们在存储库中没有任何要扫描的内容。在下一步中，我们将创建一些用于测试配置的机密。

用于测试的Secret文件

Fake.json：我们可以用它来测试JWT。

{
  "username": "SecurityBooBoo",
  "password": "Oh-Y3aH_CmG_2022-123"
}1.
2.
3.
4.

FakePrivKey.key：我们可以使用此文件来测试私钥。

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
NhAAAAAwEAAQAAAYEAvO5F4B+lb8YYXB6gCgqxlmuNLGwTC58f96TbYQxUV/yzZkBS1hIp
H0WOba5ZKeULWh7B5KPrqpK3yfYitThw73wX+UQuBN8FCyFw41/dB5brApDLLdW+aYt9M2
KU3XcKDfij3Rmw4dWvihvxeOWjp1WTNZj480O1+CB+oTOWdElWp7fmhw912itreRQcJso7
rFnfgfvdCAJoRMwYtadKmm3f9I1UvUxaaBiy45vwo7ZbnMUBnEOMhoA9zqZDii7GVj80t+
1WfbrMy/WRzibspxs4cMpUa+K9FLXoms1I35QQ+VYtFMQehjsXkZEMprfewCF0ZzmNAavL
xoIW+ybzCZNeH8afWNQ0TmO4M9sPmhrorfenKU3mfo1yxW5P8WIyl4ee7aSYPGMQdqh0nV
qAuyc2XU6YF2Qz0HpSz9qq4gQVKNS/a10QbS7KiyGtGCFyRmCV3UGHqZGo4QvGVGA20qwV
DN+o+KnLjvrYcklPBogGtxUvrCcU34n7tFE4LQjHAAAFiOcpTkznKU5MAAAAB3NzaC1yc2
EAAAGBALzuReAfpW/GGFweoAoKsZZrjSxsEwufH/ek22EMVFf8s2ZAUtYSKR9Fjm2uWSnl
C1oeweSj66qSt8n2IrU4cO98F/lELgTfBQshcONf3QeW6wKQyy3VvmmLfTNilN13Cg34o9
0ZsOHVr4ob8Xjlo6dVkzWY+PNDtfggfqEzlnRJVqe35ocPddora3kUHCbKO6xZ34H73QgC
aETMGLWnSppt3/SNVL1MWmgYsuOb8KO2W5zFAZxDjIaAPc6mQ4ouxlY/NLftVn26zMv1kc
4m7KcbOHDKVGvivRS16JrNSN+UEPlWLRTEHoY7F5GRDKa33sAhdGc5jQGry8aCFvsm8wmT
Xh/Gn1jUNE5juDPbD5oa6K33pylN5n6NcsVuT/FiMpeHnu2kmDxjEHaodJ1agLsnNl1OmB
dkM9B6Us/aquIEFSjUv2tdEG0uyoshrRghckZgld1Bh6mRqOELxlRgNtKsFQzfqPipy476
2HJJTwaIBrcVL6wnFN+J+7RROC0IxwAAAAMBAAEAAAGADHxGm+Abg754n6Xad69rOwCSjM
v7mjoC18/KO6if7kyh4nD/yGvc0dc76V2rQMyFKoh70ctPaK9Xe/5LHuTC+eCeiPeLfwDq
CWlFV5FfPwAnOb0t4DKO6dSxCnNKWTRjsraqxZLMELCZcCwWkiHC5e0O1Gzujsz7upETLT
4GhFrQYjcSAfzwkeFqsc61aY7V0LcDwUhOvBfEoj9GpsKOeJQoR7YLUpM4Kkbvk4EbrwX/
GeYfDB+eqsjQzNZKAljC346B9J/7krlANZYd/2PtsHqzJrnws/08jSjkzit12RdgP5UPWK
L1Z0et8In4vXnKUspEAagB4pGrTLdgec7s14s/RMPqkUuUv016hGzW8k14YRG9Z1e91OsA
D/kJsyLButT262kemAC05p4LUrqaAjAzDqmASSMEXfR7BWo6geVfLkxv5yDExFTcOPATNQ
tf9R62h9k+W8wHUfrxfDo8fshgOLtrSsux+ROg1tNlOsvrNrmdpPH8tZD5RQPss28xAAAA
wAZk8IIU34nLTWUspzDyyiegyAXD5kO1r6dxZoAiudaCqGjNbRo82nzrg1ux8ZohnVbOQV
t6qQkDjDQB37airXVo4/j/s7H0HthSctlEPe4EYWaouhmPk3v45LV8z8aq3QZY5chyTGB5
N17I5QgoSBHEM42pf+gWTmNUX0N4bjwRL/a+HE7PnGC5ax+FNPY+fL9fHpBtz791SogNak
fopIxmfSJJbTbBctw01F6eo67uikGOVxzk5MClJaP2jyLYFAAAAMEA6/ShdpzMCd5Oy5+O
DtDV0v8lF9YCxKl6+Vaw1iQtdafvgZuy1Msr645o39IVbUWKEwNJsi0uokyxvojcQOgNp1
hWZCVvkXXlBlTxKcxRlSBVj+6eB/nIgDLEcLKH7DYALfat+IBaEdtPiL4/+FiOmXrx/JWR
E43yoYDkC59usOZKHVl8uM4YgOw7mhFx5I1W6qX2rXoT3toEDITRygjuMjreegzhf/oSUF
T2IaGxNBpOf0yCBo6nCyFhk5f3g185AAAAwQDM+vwMD6/ToLy5roZoJ75CcVUNHMh3Sj/l
f2AEe3ldM2VTtoV/nVOli6/N8jJOhF7tRm6RlR1CjYKDPALMuRgwg83dXqtr+jxuwrcV6t
deYr48HVdTgNooJ29SmYSB+7xWX7flPS+8gtpx95FXKnNqL+NFNEWxERjfe2oLQ+2/R7Jb
XChNNNV04eIAWuCFhyl0wGBDALdo7biwsRJ/mYB7vkptfOSKTTP3zNd4qFKQsN3VBfPGmZ
KQCdxFay/Lp/8AAAATYW5hbHlzdEBweXRob25kZXYwMQ==
-----END OPENSSH PRIVATE KEY-----1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.

FakeAPIKey.txt：我们可以用它来查找和测试 API 密钥。

Apikey=33219b16f39fc40d3a1be5e9e6f56abe1.

运行测试并观察结果

默认情况下，每次添加并提交文件时，都会触发构建。如果单击 Web 界面左侧窗格中的“CI/CD”>“管道”对象，则可以看到每个作业的结果。单击每个屏幕上的“运行管道”按钮。系统将提示您输入其他可以忽略的选项。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

几秒钟后，管道作业将开始。一旦它变成蓝色，您可以单击“secret_detection”阶段并观看输出。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

如果一切按计划进行，您应该会看到类似的文本。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

下载结果

几分钟后，结果将可供下载关联作业。单击作业右侧的“下载按钮”，然后选择“secret_detection.secret_detection”文件。它将下载为 JSON。然后，您可以在自己喜欢的文本编辑器或在线 JSON 格式化工具中打开它。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

查看结果

{
  "version":"14.0.4",
  "vulnerabilities":[
    {
      "id":"a70bbb54bc6faeadc71027dfb50203a1803cc10d221af9156b5393a211ccddd2",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"password",
      "scanner":{
        "id":"gitleaks",
        "name":"Gitleaks"
      },
      "location":{
        "file":".gitlab/secret-detection-ruleset.toml",
        "commit":{
          "sha":"0000000"
        },
        "start_line":19
      },
      "identifiers":[
        {
          "type":"gitleaks_rule_id",
          "name":"Gitleaks rule ID CMG Custom Generic Password",
          "value":"CMG Custom Generic Password"
        }
      ]
    },
    {
      "id":"4d5f0e089322ca90db4e7b3baa58a3c3a68bf63af20ec9afe2e1120feea93a95",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:e45d90957eec7387726c6a1b174da7b566a24ff4cb060dcbcdfebb931a93ffe3:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"passw",
      "scanner":{
        "id":"gitleaks",
        "name":"Gitleaks"
      },
      "location":{
        "file":".gitlab/secret-detection-ruleset.toml",
        "commit":{
          "sha":"0000000"
        },
        "start_line":19
      },
      "identifiers":[
        {
          "type":"gitleaks_rule_id",
          "name":"Gitleaks rule ID CMG Custom Generic Password",
          "value":"CMG Custom Generic Password"
        }
      ]
    },
    {
      "id":"95d9614ca06da0f4fc8f67319a44585ee6295a26060cd256655ae6f503e41ef2",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:e7cf3ef4f17c3999a94f2c6f612e8a888e5b1026878e4e19398b23bd38ec221a:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"Password",
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks"
    },
      "location":{
      "file":".gitlab/secret-detection-ruleset.toml",
      "commit":{
      "sha":"0000000"
    },
      "start_line":18
    },
      "identifiers":[
      {
      "type":"gitleaks_rule_id",
      "name":"Gitleaks rule ID CMG Custom Generic Password",
      "value":"CMG Custom Generic Password"
    }
      ]
    },
      {
      "id":"52256d01ca112c54e5ca22e982b096852bf279f206f425aebf4421cfc8ac426e",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:e7cf3ef4f17c3999a94f2c6f612e8a888e5b1026878e4e19398b23bd38ec221a:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"Password",
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks"
    },
      "location":{
      "file":".gitlab/secret-detection-ruleset.toml",
      "commit":{
      "sha":"0000000"
    },
      "start_line":20
    },
      "identifiers":[
      {
      "type":"gitleaks_rule_id",
      "name":"Gitleaks rule ID CMG Custom Generic Password",
      "value":"CMG Custom Generic Password"
    }
      ]
    },
      {
      "id":"f9c6b0ef5af1fb0298ff99fc6467e76fc44a4e6d1dc3699cfd8c7249eb293ad5",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":"Fake.json:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"password",
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks"
    },
      "location":{
      "file":"Fake.json",
      "commit":{
      "sha":"0000000"
    },
      "start_line":3
    },
      "identifiers":[
      {
      "type":"gitleaks_rule_id",
      "name":"Gitleaks rule ID CMG Custom Generic Password",
      "value":"CMG Custom Generic Password"
    }
      ]
    }
      ],
      "scan":{
      "analyzer":{
      "id":"secrets",
      "name":"secrets",
      "url":"https://gitlab.com/gitlab-org/security-products/analyzers/secrets",
      "vendor":{
      "name":"GitLab"
    },
      "version":"4.5.4"
    },
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks",
      "url":"https://github.com/zricethezav/gitleaks",
      "vendor":{
      "name":"GitLab"
    },
      "version":"8.15.0"
    },
      "type":"secret_detection",
      "start_time":"2022-12-30T02:08:44",
      "end_time":"2022-12-30T02:08:44",
      "status":"success"
    }
    }1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.
99.
100.
101.
102.
103.
104.
105.
106.
107.
108.
109.
110.
111.
112.
113.
114.
115.
116.
117.
118.
119.
120.
121.
122.
123.
124.
125.
126.
127.
128.
129.
130.
131.
132.
133.
134.
135.
136.
137.
138.
139.
140.
141.
142.
143.
144.
145.
146.
147.
148.
149.
150.
151.
152.
153.
154.
155.
156.
157.
158.
159.
160.
161.
162.
163.
164.
165.
166.
167.
168.
169.
170.
171.
172.
173.
174.