如何将Secrets扫描加入到GitLab Pipeline

love374

发布于 2023-7-20 11:32

浏览

0收藏

概述

如果要管理应用程序开发管道以交付软件，则需要实施一项关键安全控制。您需要确保您的机密受到保护。简单地说，机密是您需要保护的凭据，因为该凭据具有特权功能。例如：

用户名和密码组合
接口密钥
JSON 网络令牌
私钥

开发人员经常意外地将这些凭据保存到存储库中。将这些凭据或机密添加到存储库后，清理可能会很痛苦，并且需要您更改凭据。更好的长期方法是使用“提交前检查”来避免存储机密。该主题将在另一篇文章中介绍。本博客重点介绍如何扫描现有存储库以查找 Gitlab 管道中存储的机密。

识别或创建存储库

如果您想继续操作，可以单击此链接获取公共仓库。https://gitlab.com/cmg_public/secrets_scanning_test

第一步是确定目标存储库。如果您正在入门但没有，只需在 Gitlab 中创建一个新项目即可。从菜单中单击“项目>创建新项目>导入上面项目地址”。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

扫描规则配置文件
文件路径：.gitlab/secret-detection-ruleset.toml

[secrets]
description = 'CMG DevSecOps Crash Course, secrets custom rules configuration'

[[secrets.passthrough]]
type = "raw"
target = "gitleaks.toml"
value = """\
title = "gitleaks config"
# Add regexes to the regex table
[[rules]]
description = "Test for Raw Custom Rulesets"
regex = '''Custom Raw Ruleset T[est]{3}'''

[[rules]]
description = "CMG Custom Generic Password"
regex = '''(?i)(password|passw)'''
tags = ["key", "Custom Password", "generic"]
"""

您会注意到我添加了一些自定义规则。这些将用于标识默认情况下未启用的某些类型的机密。您可以根据需要创建任意数量的规则，以查找通常出现在开发人员代码中的模式。
"CMG Generic JWT" 此正则表达式（REGEX）查找存储在文件中的 JSON Web Token。诸如“JWT”、“jwt_token”、“secret”、“bearer”等。正则表达式不区分大小写。

[[rules]]
description = "CMG Generic JWT"
regex = '''(?i)(jwt|jwt_token|secret|bearer)'''
tags = ["key", "JWT", "generic"]

"CMG Custom Generic Password" 此正则表达式查找存储在文件中的单词“password”的常见派生词。此正则表达式也不区分大小写。

[[rules]]
description = "CMG Custom Generic Password"
regex = '''(?i)(password|passw)'''
tags = ["key", "Custom Password", "generic"]
"""

Pipeline配置文件

此模板将用于根据默认设置和任何自定义正则表达式字符串（如我们在前面的步骤中添加的字符串）扫描机密。

# This file is a template, and might need editing before it works on your project.
# This template moved to Jobs/Secret-Detection.gitlab-ci.yml in GitLab 14.0
# Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/292977
include:
  template: Jobs/Secret-Detection.gitlab-ci.yml

此时，我们已准备好运行管道;但是，我们在存储库中没有任何要扫描的内容。在下一步中，我们将创建一些用于测试配置的机密。

用于测试的Secret文件

Fake.json：我们可以用它来测试JWT。

{
  "username": "SecurityBooBoo",
  "password": "Oh-Y3aH_CmG_2022-123"
}

FakePrivKey.key：我们可以使用此文件来测试私钥。

FakeAPIKey.txt：我们可以用它来查找和测试 API 密钥。

Apikey=33219b16f39fc40d3a1be5e9e6f56abe

运行测试并观察结果

默认情况下，每次添加并提交文件时，都会触发构建。如果单击 Web 界面左侧窗格中的“CI/CD”>“管道”对象，则可以看到每个作业的结果。单击每个屏幕上的“运行管道”按钮。系统将提示您输入其他可以忽略的选项。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

几秒钟后，管道作业将开始。一旦它变成蓝色，您可以单击“secret_detection”阶段并观看输出。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

如果一切按计划进行，您应该会看到类似的文本。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

下载结果

几分钟后，结果将可供下载关联作业。单击作业右侧的“下载按钮”，然后选择“secret_detection.secret_detection”文件。它将下载为 JSON。然后，您可以在自己喜欢的文本编辑器或在线 JSON 格式化工具中打开它。

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

查看结果

{
  "version":"14.0.4",
  "vulnerabilities":[
    {
      "id":"a70bbb54bc6faeadc71027dfb50203a1803cc10d221af9156b5393a211ccddd2",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"password",
      "scanner":{
        "id":"gitleaks",
        "name":"Gitleaks"
      },
      "location":{
        "file":".gitlab/secret-detection-ruleset.toml",
        "commit":{
          "sha":"0000000"
        },
        "start_line":19
      },
      "identifiers":[
        {
          "type":"gitleaks_rule_id",
          "name":"Gitleaks rule ID CMG Custom Generic Password",
          "value":"CMG Custom Generic Password"
        }
      ]
    },
    {
      "id":"4d5f0e089322ca90db4e7b3baa58a3c3a68bf63af20ec9afe2e1120feea93a95",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:e45d90957eec7387726c6a1b174da7b566a24ff4cb060dcbcdfebb931a93ffe3:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"passw",
      "scanner":{
        "id":"gitleaks",
        "name":"Gitleaks"
      },
      "location":{
        "file":".gitlab/secret-detection-ruleset.toml",
        "commit":{
          "sha":"0000000"
        },
        "start_line":19
      },
      "identifiers":[
        {
          "type":"gitleaks_rule_id",
          "name":"Gitleaks rule ID CMG Custom Generic Password",
          "value":"CMG Custom Generic Password"
        }
      ]
    },
    {
      "id":"95d9614ca06da0f4fc8f67319a44585ee6295a26060cd256655ae6f503e41ef2",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:e7cf3ef4f17c3999a94f2c6f612e8a888e5b1026878e4e19398b23bd38ec221a:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"Password",
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks"
    },
      "location":{
      "file":".gitlab/secret-detection-ruleset.toml",
      "commit":{
      "sha":"0000000"
    },
      "start_line":18
    },
      "identifiers":[
      {
      "type":"gitleaks_rule_id",
      "name":"Gitleaks rule ID CMG Custom Generic Password",
      "value":"CMG Custom Generic Password"
    }
      ]
    },
      {
      "id":"52256d01ca112c54e5ca22e982b096852bf279f206f425aebf4421cfc8ac426e",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":".gitlab/secret-detection-ruleset.toml:e7cf3ef4f17c3999a94f2c6f612e8a888e5b1026878e4e19398b23bd38ec221a:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"Password",
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks"
    },
      "location":{
      "file":".gitlab/secret-detection-ruleset.toml",
      "commit":{
      "sha":"0000000"
    },
      "start_line":20
    },
      "identifiers":[
      {
      "type":"gitleaks_rule_id",
      "name":"Gitleaks rule ID CMG Custom Generic Password",
      "value":"CMG Custom Generic Password"
    }
      ]
    },
      {
      "id":"f9c6b0ef5af1fb0298ff99fc6467e76fc44a4e6d1dc3699cfd8c7249eb293ad5",
      "category":"secret_detection",
      "name":"CMG Custom Generic Password",
      "message":"CMG Custom Generic Password detected; please remove and revoke it if this is a leak.",
      "description":"CMG Custom Generic Password",
      "cve":"Fake.json:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8:CMG Custom Generic Password",
      "severity":"Critical",
      "confidence":"Unknown",
      "raw_source_code_extract":"password",
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks"
    },
      "location":{
      "file":"Fake.json",
      "commit":{
      "sha":"0000000"
    },
      "start_line":3
    },
      "identifiers":[
      {
      "type":"gitleaks_rule_id",
      "name":"Gitleaks rule ID CMG Custom Generic Password",
      "value":"CMG Custom Generic Password"
    }
      ]
    }
      ],
      "scan":{
      "analyzer":{
      "id":"secrets",
      "name":"secrets",
      "url":"https://gitlab.com/gitlab-org/security-products/analyzers/secrets",
      "vendor":{
      "name":"GitLab"
    },
      "version":"4.5.4"
    },
      "scanner":{
      "id":"gitleaks",
      "name":"Gitleaks",
      "url":"https://github.com/zricethezav/gitleaks",
      "vendor":{
      "name":"GitLab"
    },
      "version":"8.15.0"
    },
      "type":"secret_detection",
      "start_time":"2022-12-30T02:08:44",
      "end_time":"2022-12-30T02:08:44",
      "status":"success"
    }
    }

查看漏洞报告

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

这个例子找到了我们的“Fake.Json”文件。在详细信息页面上，单击"Fake.json:3". 该工具会将您定向到找到的特定文件和凭据：

如何将Secrets扫描加入到GitLab Pipeline-鸿蒙开发者社区

总结

总之，从安全角度来看，您可以看到这是简单且有价值的。一旦你的秘密被提交，修复起来就是一个痛苦的过程。这是一种方法，但不会否定对凭据和机密存储进行“预提交扫描”或使用“vault”的需要。

文章转载自公众号：DevOps云学堂

分类

其他

标签

GitLab Pipeline

已于2023-7-20 11:32:49修改

51CTO

51CTO博客

51CTO学堂

如何将Secrets扫描加入到GitLab Pipeline

概述

识别或创建存储库

用于测试的Secret文件

运行测试并观察结果

下载结果

总结

目录

订阅鸿蒙技术特刊，精选内容抢先看