Use a maintained tool for preventing checking secrets into version control

[afeld]

User Story:

As someone in TTS, I want to use a maintained tool for secrets.

Problem Statement:

We use a fork of git-seekret which isn't maintained. We should investigate using git-secrets or something like it.

Background information:

Actions to take:

• Look up tools for secrets
• Look up git-secrets
• Look up git-seekret

Acceptance criteria:

• Propose path forward on managing secrets in git - (See comments below - a solution has been proposed circulating for agreement {{ DRAFT 3/2/2020 }})
• Implement path forward on managing secrets in git

Supporting Documentation:

git-seekret
git-secrets

Related Issues:

#184
#189
https://github.com/18F/tts-tech-portfolio/issues/91

[LinuxBozo]

@afeld since that was mostly my doing, here’s the deets. At the time, there was not an easy way to install git-secrets without a bunch of dependency hell (wasn’t installable from brew for example). Also, the output is less than ideal for non-developers, giving you no real clue what you were violating unless you could read/understand the regex, and this was something we were instituting across the org for all staff. We also thought we were going to get more support from the original developer, since was fairly responsive to PRs at first, and then he just stopped. We still have PRs out there that went unmerged.

As for output, compare git-secrets:

/tmp/example:3:password=******

[ERROR] Matched prohibited pattern

Possible mitigations:
- Mark false positives as allowed using: git config --add secrets.allowed ...
- List your configured patterns: git config --get-all secrets.patterns
- List your configured allowed patterns: git config --get-all secrets.allowed
- Use --no-verify if this is a one-time false positive

VS. what is output from git-seekret

Found Secrets: 1
        /tmp/example:3
                - Metadata:
                  status: test
                - Rule:
                  password.password
                - Content:
                  password = 'this is super secret'

[LinuxBozo]

Why is that subtle difference important you may be asking?
I can give each rule a name with git-seekret, which can add additional context on the issue, and I can group the exceptions with the rule definition. With git-secrets the rules are just a line separated file with a bunch of regexes.

Let's take searching for New Relic license keys.

git-seekret I can define the rule in it's own file, and when it's matched, I see the Rule comes up as newrelic.license_key when it's found:

license_key:
  match: (\"|')?(NEW|New|new)?_?(RELIC|Relic|relic)_?(LICENSE|License|license)?_?(KEY|Key|key)?\s*(:|=>|=)?\s*(\"|')?[0-9A-Fa-f]{40}(\"|')?
  unmatch:
    - (\"|')?github.com.*/#[A-Za-z0-9]{40}(\"|')?
    - (\"|')?sha(\"|')?\s*(:|=>|=)\s*(\"|')?[A-Za-z0-9]{40}(\"|')?
    - (\"|')?revision(\"|')?\s*(:|=>|=)\s*(\"|')?[A-Za-z0-9]{40}(\"|')?
    - (\"|')?version(\"|')?\s*(:|=>|=)\s*(\"|')?[A-Za-z0-9]{40}(\"|')?
    - (\"|')?hash(\"|')?\s*(:|=>|=)\s*(\"|')?[A-Za-z0-9]{40}(\"|')?

[hillaryj]

Could we re-prioritize this? cloud.gov has had two incidents recently with credentials being committed and we'd love to have a working tool to minimize the chance of more of these. Thanks!

• https://github.com/18F/security-incidents/issues/194
• https://github.com/18F/security-incidents/issues/195

[pburkholder]

Is git seekrets actually broken for folks, or just not tested/trusted? I think the UI argument raised on the issue matters not a whit when it’s just not working.

[mgwalker]

@hillaryj I'm curious if updating the git-seekrets rules could prevent a repeat of those.

I suspect the rules are going to be an issue regardless of the tool, though maybe the other tools have built-in rules that already cover our needs. I'd still be interested in seeing how they perform against issues like the ones Hillary mentioned.

I second @pburkholder's question. If the tool actually works and it's an issue of outdated/untested rules, then changing tools only helps if the new tool also has the rules we need and those are maintained and tested. If we still have to write and maintain our own rules, then a different tool isn't going to be any better.

[mgwalker]

Sorry, realized I forgot to include a link to our current rules:
https://github.com/18F/laptop/tree/master/seekret-rules