CredSweeper
Table of Contents
Introduction
CredSweeper is a tool to detect credentials in any directories or files. CredSweeper could help users to detect unwanted exposure of credentials (such as personal information, token, passwords, api keys and etc) in advance. By scanning lines, filtering, and using AI model as option, CredSweeper reports lines with possible credentials, where the line is, and expected type of the credential as a result.
How To Use
Main Requirements
- Python3.7 or higher
Installation
Via pip
Stable (release 1.0.0):
$ pip install git+https://github.com/Samsung/[email protected]
Latest (current master branch):
$ pip install git+https://github.com/Samsung/CredSweeper.git
Via git clone (dev install)
$ git clone https://github.com/Samsung/CredSweeper.git
$ cd CredSweeper
$ pip install -qr requirements.txt
Run
Get all argument list:
$ python -m credsweeper --help
usage: python -m credsweeper [-h] --path PATH [PATH ...] [--rules [PATH]] [--ml_validation] [--api_validation] [-j POSITIVE_INT] [--skip_ignored] [--save-json [PATH]] [--log LOG_LEVEL]
optional arguments:
-h, --help show this help message and exit
--path PATH [PATH ...]
file or directory to scan
--rules [PATH] path of rule config file (default: credsweeper/rules/config.yaml)
--ml_validation ml validation option on
--api_validation api validation option on
-j POSITIVE_INT, --jobs POSITIVE_INT
number of parallel processes to use (default: number of CPU cores * 2)
--skip_ignored parse .gitignore files and skip credentials from ignored objects
--save-json [PATH] save result to json file (default: output.json)
--log [LOG_LEVEL] set logging level. Example --log debug, (default: 'warning'),
detailed log config: credsweeper/secret/log.yaml
Get output as JSON file:
$ python -m credsweeper --ml_validation --path tests/samples/password --save-json output.json
rule: Password / severity: medium / line_data_list: [line : 'password = "cackle!"' / line_num : 1 / path : tests/samples/password / entropy_validation: False] / api_validation: NOT_AVAILABLE / ml_validation: VALIDATED_KEY
$ cat output.json
[
{
"rule": "Password",
"severity": "medium",
"line_data_list": [
{
"line": "password = \"cackle!\"",
"line_num": 1,
"path": "tests/samples/password",
"entropy_validation": false
}
],
"api_validation": "NOT_AVAILABLE",
"ml_validation": "VALIDATED_KEY"
}
]
Get CLI output only:
$ python -m credsweeper --ml_validation --path tests/samples/password
rule: Password / severity: medium / line_data_list: [line : 'password = "cackle!"' / line_num : 1 / path : tests/samples/password / entropy_validation: False] / api_validation: NOT_AVAILABLE / ml_validation: VALIDATED_KEY
Tests
To run all tests:
$ python -m pytest --cov=credsweeper --cov-report=term-missing -s tests/
To run only tests independent from external api:
$ python -m pytest -m "not api_validation" --cov=credsweeper --cov-report=term-missing -s tests/
Benchmark
We have a dataset for testing credential scanners that called CredData. If you want to test CredSweeper with this dataset please check here.
Overall Architecture
To check overall architecture of CredSweeper please check here.
License
The CredSweeper is an Open Source project released under the terms of MIT License V2.
How to Get Involved
In addition to developing under an Open Source license, A use an Open Source Development approach, welcoming everyone to participate, contribute, and engage with each other through the project.
Project Roles
A recognizes the following formal roles: Contributor and Maintainer. Informally, the community may organize itself and give rights and responsibilities to the necessary people to achieve its goals.
Contributor
A Contributor is anyone who wishes to contribute to the project, at any level. Contributors are granted the following rights, to:
- Contribute code, documentation, translations, artwork, and etc.
- Report defects (bugs) and suggestions for enhancement.
- Participate in the process of reviewing contributions by others.
Contributors are required to:
- Receive the approval of the maintainer to contribute and reflect changes.
- Assume responsibility for issues and bugs introduced by one's own contributions.
- Provide constructive advice whenever participating in discussions and in the review of contributions.
- Check code formatting before commit. The CredSweeper use yapf and to automate code formatting, run commands below.
pip install pre-commit pre-commit install
Contributors who show dedication and skill are rewarded with additional rights and responsibilities. Their opinions weigh more when decisions are made, in a fully meritocratic fashion.
Maintainer
A Maintainer is a Contributor who is also responsible for knowing, directing and anticipating the needs of a given a Module. As such, Maintainers have the right to set the overall organization of the source code in the Module, and the right to participate in the decision-making. Maintainers are required to review the contributor’s requests and decide whether to accept or not.
| Name | |
|---|---|
| Jaeku Yun | [email protected] |
| Shinhyung Choi | [email protected] |
| Yujeong Lee | [email protected] |
| Oleksandra Sokol | [email protected] |
How to Contact
Please post questions, issues, or suggestions into Issues, This is the best way to communicate with the developer.
343 Jan 02, 2023
2 Oct 18, 2021
4 Sep 08, 2022
2 Oct 13, 2021
7 Nov 17, 2022
948 Dec 31, 2022
31 Nov 17, 2022
1 Jan 30, 2022
37 Nov 30, 2022
2.1k Jan 01, 2023
1 Oct 17, 2021
2 Nov 14, 2021
9 Mar 21, 2022
533 Dec 29, 2022
731 Dec 27, 2022
2 Nov 28, 2021
206 Jan 02, 2023
15 Dec 25, 2022
1 Dec 09, 2021
10 Aug 31, 2022