Find out if your GitHub repository contains sensitive information such as password, secret keys, confidential information, etc.
GitHub is used by millions of users to host and share codes. It’s fantastic, but sometimes you/developers/code owners can accidentally dump confidential information in a public repository, which can be a disaster.
There are many incidents where confidential data was leaked on GitHub. You can’t eliminate human error but can take action to reduce that.
How do you ensure your repository doesn’t contain a password or key?
Simple answer – don’t store.
As a best practice, one should use secret management software to store all sensitive information.
But in reality, you can’t control other people’s behaviour if working in a team.
BTW, if you use Git to initialize and deploy your application, it creates .git a folder, and if accessible over the Internet, it may expose sensitive confirmation – which you don’t want and should consider blocking .git URI.
The following solutions help you to find mistakes in your repository.
Secret Scanning
GitHub’s secret scanning feature is a powerful tool that detects accidental secrets hidden in your code, safeguarding against data leaks and compromises. It works seamlessly for both public and private repositories, meticulously combing through every nook and cranny to uncover any slipped secrets.
But its capabilities continue. Once a secret is discovered, GitHub takes proactive measures by alerting the respective service providers, prompting them to mitigate any potential risks swiftly. Regarding private repositories, GitHub goes the extra mile by notifying organization owners or administrators, ensuring that the right individuals within your team are immediately made aware of the situation.
To provide continuous visibility, warnings are prominently displayed within the repository, serving as a clear signal for you and your team to take prompt action. GitHub’s secret scanning feature acts as your vigilant ally, diligently working to ensure that no secret goes unnoticed and your projects remain secure.
Embrace the power of secret scanning and code confidently, knowing your sensitive information is protected.
Git Secrets
Let me introduce you to git-secrets, a tool that can save us from the embarrassment of accidentally adding secrets to our Git repositories. It scans commits, commit messages, and merges to prevent secret leakage in our code.
To get started on Windows, we simply run the install.ps1 PowerShell script. It copies the necessary files to an installation directory and adds them to our user PATH. This makes git-secrets easily accessible from anywhere in our development environment.
Once installed, git-secrets becomes our watchful guardian, checking if any commit, commit message, or merge history matches our configured prohibited patterns. If it detects a match, it rejects the commit, preventing sensitive information from slipping through the cracks.
We can add regular expression patterns to a .gitallowed file in the repository’s root directory to fine-tune git-secrets. This helps filter out any lines that might trigger a warning but are legitimate, striking the right balance between security and convenience.
When scanning a file, git-secrets extract all lines that match prohibited patterns and provide detailed information, including file paths, line numbers, and the matched lines. It also checks whether the matched lines match our registered allowed patterns. The commit or merge is considered safe if allowed patterns cancel out all flagged lines. However, git-secrets block the process if any matched lines don’t match an allowed pattern.
While using git-secrets, we need to be cautious. Prohibited patterns should not be too broad, and allowed patterns should not be too permissive. Testing our patterns using ad-hoc calls to git secrets –scan $filename ensures they work as intended.
If you’re eager to dive deeper into git-secrets or want to contribute to its development, you’ll find the project on GitHub. It’s an open-source project that encourages community contributions. Join the community and make a difference!
With git-secrets, we can code confidently, knowing that accidental secrets won’t jeopardize our projects. Let’s embrace this tool and keep our sensitive information secure.
Repo Supervisor
I’ve got this exciting news: Repo-supervisor is a powerful tool that detects secrets and passwords in your code. Installing it is a breeze—simply add a webhook to your GitHub repository. Repo-supervisor offers two modes: scanning pull requests on GitHub or scanning local directories from the command line. Choose the mode that suits you best.
To embark on your git-secrets journey, simply visit the GitHub repository and download the most recent release. There, you’ll discover bundles tailored for AWS Lambda deployment and a user-friendly CLI mode. With the CLI mode, you can dive right in without any extra setup, while the pull request mode necessitates deploying to AWS Lambda. Choose the option that suits your needs and start harnessing the power of git-secrets to bolster the security of your codebase!
In CLI mode, provide a directory as an argument, and Repo-supervisor scans supported file types, processing each file with a tokenizer specific to its type. It runs security checks on extracted strings and provides clear reports in plain text or JSON format.
For pull request mode, Repo-supervisor processes webhook payloads, extracts modified files, and performs security checks on the extracted strings. If issues are found, it sets the CI status to error, linking to the report. No issues mean a successful CI status.
Repo-supervisor is an amazing code inspector that keeps our secrets and passwords secure. It ensures the integrity of our codebase, which is crucial in our professional lives.
Give Repo-supervisor a try! Install it, configure the webhook, and let it scan for secrets and passwords. Enjoy the extra layer of security!
Truffle Hog
Allow me to introduce you to an incredible tool called Truffle Hog. Consider it your loyal code companion, diligently sniffing out any trace of sensitive information lurking within your repositories. Truffle Hog is a master at delving deep into your project’s history, meticulously scanning for potential leaks of valuable secrets such as API keys and passwords.
With its arsenal of high-entropy checks and regex patterns, this tool is primed to unearth these hidden treasures and ensure your code remains secure. Say goodbye to secret leaks and embrace the vigilant protection of Truffle Hog!
And here’s the best part: Truffle Hog’s latest version is packed with many new powerful features. It now boasts over 700 credential detectors that actively verify against their respective APIs. It also supports scanning GitHub, GitLab, filesystems, S3, GCS, and Circle CI, making it incredibly versatile.
Not only that, TruffleHog now has native support for instant verification of private keys against millions of GitHub users and billions of TLS certificates using its cutting-edge Driftwood technology. It can even scan binaries and other file formats, ensuring no stone is left unturned.
What’s more, TruffleHog is available as both a GitHub Action and a pre-commit hook, seamlessly integrating into your development workflow. It’s designed to be convenient and user-friendly, providing an extra layer of security without causing unnecessary hassle.
With Truffle Hog in your toolkit, you can confidently protect your code from accidental exposures and keep your secrets locked away. So give Truffle Hog a try, and let it work its magic in safeguarding your projects.
Git Hound
GitHound goes beyond the limitations of other tools by leveraging GitHub code search, pattern matching, and commit history searching. It can search the entire GitHub, not just specific repositories, users, or organizations. How cool is that?
Now, let’s dive into its fantastic features. Git Hound utilizes GitHub/Gist code search, allowing it to pinpoint sensitive information scattered across the vast expanse of GitHub, uploaded by anyone. It’s like having a treasure map to uncover potential vulnerabilities.
But GitHound doesn’t stop there. It detects sensitive data by employing pattern matching, contextual information, and string entropy. It even digs deep into commit history to find improperly deleted secrets, ensuring no stone is left unturned.
To simplify your life, GitHound incorporates a scoring system that filters out common false positives and optimizes its search for intensive repository digging. It’s designed to save you time and effort.
And guess what? Git Hound is equipped with base64 detection and decoding capabilities. It can unveil hidden secrets encoded in base64 format, giving you an extra edge in your hunt for sensitive information.
What’s more, GitHound offers options to integrate it into larger systems. You can generate JSON output and customize regexes according to your specific needs. It’s all about flexibility and empowering you to build upon its foundation.
Now, let’s talk about its exciting use cases. In the corporate world, GitHound becomes invaluable in searching for exposed customer API keys. It helps safeguard sensitive information, ensuring the highest level of security.
For bug bounty hunters, Git Hound is a game-changer. It enables you to search for leaked employee API tokens, helping you uncover vulnerabilities and earn those well-deserved bounties. Isn’t Git Hound amazing?
Gitleaks
Gitleaks is designed to make your life easier. It’s an easy-to-use, all-in-one solution that detects secrets, whether they’re buried in your code’s past or present. Say goodbye to the risk of exposing passwords, API keys, or tokens in your projects.
Installing Gitleaks is a breeze. You can use Homebrew, Docker, or Go, depending on your preference. Plus, it offers flexible implementation options. You can set it up as a pre-commit hook directly in your repository or take advantage of Gitleaks-Action to integrate it seamlessly into your GitHub workflows. It’s all about finding the setup that suits you best.
Now, let’s talk about the commands that Gitleaks offers. First, we have the “detect” command. This powerful command enables you to scan repositories, directories, and individual files. Whether working on your own machine or in a CI environment, Gitleaks has got you covered. It ensures that no secret slips through the cracks.
But that’s not all. Gitleaks also provides the “protect” command. This command scans explicitly uncommitted changes in your Git repositories. It acts as your last line of defense, preventing secrets from being inadvertently committed. It’s a safeguard that keeps your code clean and secure.
Tines, a trusted name in the industry, sponsors Gitleaks. With their support, Gitleaks continues to evolve and improve, providing you with the best secret detection capabilities.
So, my young professionals, don’t let secrets compromise your projects. Install it, set it up, and let it do the hard work of scanning and protecting your repositories
Repo Security Scanner
The repo security scanner is an invaluable command-line tool designed to aid in the identification of inadvertently committed sensitive data, such as passwords, tokens, private keys, and other secrets, within your Git repository.
This powerful tool empowers you to proactively detect and address potential security vulnerabilities arising from the unintentional inclusion of confidential information in your codebase. By employing the repo security scanner, you can ensure the integrity of your repository and safeguard your sensitive data from unauthorized access.
Repo Security Scanner effortlessly delves into the entire repository’s history, swiftly presenting comprehensive scan results. By conducting thorough scans, it empowers you to proactively identify and swiftly address potential security vulnerabilities that may arise from exposed secrets in open-source software.
Git Guardian
GitGuardian is a tool that enables developers, security, and compliance teams to monitor GitHub activity in real time and identify vulnerabilities due to exposed secrets like API tokens, security certificates, database credentials, etc.
GitGuardian allows the teams to enforce security policies in private and public code and other data sources.
GitGuardian major features are;
- The tool helps to find sensitive information, such as secrets in the private source code,
- Identify and fix sensitive data leaks on public GitHub.
- It is an effective, transparent, and easy-to-set-up secret detection tool.
- Wider coverage and comprehensive database to cover almost any sensitive information at risk.
- Sophisticated pattern-matching techniques that improve the discovery process and effectiveness.
Conclusion
I hope this gives you an idea of finding sensitive data in the GitHub repository. If you are using AWS, then check out this article to scan AWS security and misconfiguration. Stay tuned for more exciting tools that will enhance your professional life. Happy coding, and keep those secrets locked away! 🔒