C/C++ Dependency Analyzer: a rewrite of John Lakos' dep_utils (adep/cdep/ldep) from

Overview

logo

https://travis-ci.org/rakhimov/cppdep.svg?branch=master 'Build status' Code Health

cppdep performs dependency analysis among components/packages/package groups of a large C/C++ project. This is a rewrite of dep_utils(adep/cdep/ldep), which is provided by John Lakos' book "Large-Scale C++ Software Design", Addison Wesley (1996).

Limitations

  • Indirect extern declarations of global variables or functions instead of including the proper component header with the declarations.
  • Embedded dynamic dependencies, such as dynamic loading and configurable internal services.
  • Preprocessing or macro expansion is not performed. Dependency inclusion via preprocessor meta-programming is not handled.
  • Dependency exclusion with C style multi-line comments or macros is not respected.

Requirements

  1. Python 2.7 or 3.4+
  2. NetworkX
  3. pydot
  4. pydotplus
  5. PyYAML
  6. PyKwalify 1.6.0+

The dependencies can be installed with pip.

$ sudo pip install -r requirements.txt

Installation

From the source:

$ ./setup.py install

The latest stable release from PyPi:

$ pip install cppdep

Usage

Create a configuration file that describes the project for analysis. config_schema.yml is given for guidance.

In the root directory of the project with the configuration file, run the following command to generate dependency analysis reports and graphs.

$ cppdep -c /path/to/config/file

More documentation and example configurations can be found in project wiki.

Acknowledgments

  • John Lakos for inventing the analysis and providing dep_utils.
  • Zhichang Yu for rewriting dep_utils into Python.
Comments
  • Behavior specification for anomalous conflicting component files

    Behavior specification for anomalous conflicting component files

    Anomalous (rare, error) case of having "component.c" and "component.cc" or "component.h" and "component.hpp" at the same time trying to define the same component. In other words, files with the same basenames but different extensions in header and/or source groups.

    opened by rakhimov 1
  • Missing 'pydot' dependency fails on networkx nx_pydot

    Missing 'pydot' dependency fails on networkx nx_pydot

    Traceback (most recent call last):
      File "/.../python3env/bin/cppdep", line 11, in <module>
        sys.exit(main())
      File "/.../python3env/lib/python3.5/site-packages/cppdep/__main__.py", line 59, in main
        analysis.analyze(printer, args)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 797, in analyze
        lambda x: isinstance(x, PackageGroup)))
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 779, in _analyze
        digraph.write_dot(graph_name)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/graph.py", line 273, in write_dot
        write_dot(self.digraph, file_basename + '.dot')
      File "<decorator-gen-416>", line 2, in write_dot
      File "/.../python3env/lib/python3.5/site-packages/networkx/utils/decorators.py", line 224, in _open_file
        result = func(*new_args, **kwargs)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 54, in write_dot
        P = to_pydot(G)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 199, in to_pydot
        pydot = _import_pydot()
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 348, in _import_pydot
        import pydot
    
    bug 
    opened by rakhimov 0
  • Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    If component/project names contain paths as identifiers, the path separator on Windows \ followed by 'U' or 'u' is interpreted as a Unicode literal by pydotplus plotting on Windows with python 2.7.

    This problem does not show up on python 3.

    For the sake of consistency in the report (stable report) and testing, use Unix path separator in Ids.

    bug 
    opened by rakhimov 0
  • Handle 'ipp' template implementation/source files

    Handle 'ipp' template implementation/source files

    Even though templates are only in headers, ipp files could be considered as implementation pair of the interface header. This occurs, for example, in Boost. #29.

    opened by rakhimov 0
  • Redundant 'ldep' printing of cumulative dependencies

    Redundant 'ldep' printing of cumulative dependencies

    Unlike the original ldep, the cppdep is printing the whole cumulative dependencies; that is, the all link time dependencies are printed. This information is visually provided in the graph, and implicitly provided in the original ldep '-l | -L' flags.

    It seems like the current implementation of the cumulative dependency printing is geared towards debugging rather than analysis report.

    Fix this printing by implementing the original behavior of ldep.

    bug 
    opened by rakhimov 0
  • Pairing header and implementation files in different locations

    Pairing header and implementation files in different locations

    Header and implementation files can be located in different directories, e.g., headers in include and implementation in src. The current implementation assumes the same location.

    bug enhancement 
    opened by rakhimov 0
  • Deduce external packages from the include directives w/o filesystem search

    Deduce external packages from the include directives w/o filesystem search

    The package can be deduced from the include directives following the pattern: "<package/header>", e.g., "<boost/any.hpp>". This approach will avoid the relatively expensive lookup of the header from the system. Whether such external header actually exists on the system or not is irrelevant to the dependency analysis.

    This heuristic would make the configuration simpler and more robust for cross-platform work.

    In addition, the standard library headers can be "hard-coded" into the script since these are the most likely to be used/searched by the source files.

    There's a small chance of false positives if the project under analysis happens not to conform to this convention and invents its own packages with the same names as external ones, e.g., boost, qt, libxml.

    enhancement 
    opened by rakhimov 0
  • Double counting of common components in CCD

    Double counting of common components in CCD

    The current implementation overcounts link time dependencies with common components deep in the dependency graph. Consider: A->B->C, A->D->C. CCD(A) must be 4 (4 object files in total.) The current approach overcounts the common object C, so CCD(A) ends up being 5.

    bug 
    opened by rakhimov 0
  • Extended definition for 'Component'

    Extended definition for 'Component'

    Even though John Lakos defined a component as a pair of h and c files, C++ can have template only components residing only in header files (e.g., STL/Boost/etc.). Moreover, some header-only components may contain only inline functions or macros without any need for an implmentation file (e.g., inline math, Boost PPL). For these reason, unpaired header files can be counted as components by default.

    In addition, the implementation file containing the main function of an application could be considered as a component as well (an entry point).

    bug enhancement 
    opened by rakhimov 0
  • Incorrect dependency processing with file basenames

    Incorrect dependency processing with file basenames

    The current implementation uses the basename of the included files as the key in dependency search. This approach ignores the current location of the dependent files and the language include rules, i.e. <header.h> vs. "header.h". Moreover, it results in name conflicts for files that may belong to completely separate projects or packages.

    bug 
    opened by rakhimov 0
  • Compatibility fixes

    Compatibility fixes

    This PR contains some fixes that I needed to get the cppdep tool running in my recent python 3.8 environment. Also handling of errors in the yaml file was adapted a little bit to get more helpful error messages

    opened by jsinge 0
  • Missing graph module

    Missing graph module

    I've just installed this project, tried to run it but I get this error message.

    Traceback (most recent call last):
      File "cppdep.py", line 36, in <module>
        from .graph import Graph
    ModuleNotFoundError: No module named '__main__.graph'; '__main__' is not a package
    
    bug help wanted question 
    opened by nbourre 2
  • Python 3.6 os.path.commonprefix expects List

    Python 3.6 os.path.commonprefix expects List

      File "/.../cppdep/cppdep.py", line 92, in path_common
        path = os.path.commonprefix(paths)
      File "/home/olzhas/temp/pyenv/lib/python3.6/genericpath.py", line 76, in commonprefix
        if not isinstance(m[0], (list, tuple)):
    TypeError: 'set' object does not support indexing
    
    bug 
    opened by rakhimov 1
  • Include Wrangler Analysis

    Include Wrangler Analysis

    Incorporation of include-wrangler analysis

    • header file cost
    • include cost
    • translation unit cost
    • other features

    In collaboration with include-wrangler author @lukedodd

    enhancement 
    opened by rakhimov 0
  • Options to disable/enable warnings

    Options to disable/enable warnings

    • [ ] Duplicate include
    • [ ] Redundant (transitive from the component header) include
    • [ ] Missing a component header (incomplete component)
    • [ ] Failure to locate a header from an include directive
    enhancement 
    opened by rakhimov 0
Releases(0.2.0)
  • 0.2.0(Feb 3, 2017)

    Added

    • Pairing header and implementation files in different locations (#19)
    • Handle 'ipp' template implementation source files (#31)
    • Behavior specification for anomalous conflicting component files (#27)
    • Implement ignore/exclude paths (#23)
    • Accept glob pattern for source paths (#36)
    • Project wiki pages
    • Regex pattern based include directive classification (#22)
    • Deduce external packages from the include directive w/o filesystem search (#18)
    • Handle header files w/o extensions (Boost/STL/Qt/etc.) (#32)
    • Use POSIX path separator in component names (for cross-platform report stability)
    • Configuration file validation against the schema (with PyKwalify)

    Changed

    • pytest instead of nose
    • YAML configuration files instead of XML (#24)

    Removed

    • Implicit single-path alias Package construction

    Fixed

    • Exception leaks out of main()
    • Unicode Escape Error on graph dot on Windows with Python 2.7 (#35)
    • Python3 UnicodeDecodeError for 'utf-8' in source files (#30)
    • Logging: Type Error: not all arguments converted during string formatting (#28)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 6, 2017)

    Added

    • The original ldep '-l|-L' options to print dependencies (#20)
    • '-o' to print reports into a file
    • Warn about duplicate and redundant includes (#13)
    • Extended definition for 'Component' (#7)
    • PEP-257 conformance (#2)
    • PEP-8 conformance (#1)
    • Python 3 support
    • PyPI package
    • XML configuration example and RNG schema
    • Travis CI (Linux, OS X) and AppVeyor CI (Windows) setups

    Changed

    • Differentiate 'paths' into source, include, and alias.
    • Print warnings to stderr instead of stdout (#12)
    • Report Component levels instead of Graph layers (#9)
    • Refactor the procedural design into the object-oriented design (#4)
    • Change '-f' flag into '-c' flag
    • Replace optparse with argparse
    • XML configuration file format

    Removed

    • Redundant printing a list of cumulative dependencies (#20)
    • Indirect missing-header include warnings
    • Global cross-package and cross-package-group component dependency analysis
    • 'details-of-components/--debug' verbosity
    • dot2any.py helper script
    • Manual profiling code (use pyvmmonitor instead)
    • Manual testing code (automated with nosetest)

    Fixed

    • Level 0 External components missing from the report and graph (#21)
    • Incorrect dependency processing with file basenames (#6)
    • Wrong level calculation for cycles (#8)
    • Double counting of common components in CCD calculations (#11)
    • Missing cycles from the Dot graph (#10)
    • Outdated networkx API usage
    Source code(tar.gz)
    Source code(zip)
Owner
Olzhas Rakhimov
Olzhas Rakhimov
C/C++ Dependency Analyzer: a rewrite of John Lakos' dep_utils (adep/cdep/ldep) from

Version bêta d'un système pour suivre les prix des livres chez Books to Scrape, un revendeur de livres en ligne. En pratique, dans cette version bêta, le programme n'effectuera pas une véritable surv

Olzhas Rakhimov 125 Sep 21, 2022
Calculator Python Package

Calculator Python Package This is a Calculator Package of Python. How To Install The Package? Install packagearinjoyn with pip (Package Installer Of P

Arinjoy_Programmer 1 Nov 21, 2021
ticktock is a minimalist library to profile Python code

ticktock is a minimalist library to profile Python code: it periodically displays timing of running code.

Victor Benichoux 30 Sep 28, 2022
Collects all accepted (partial and full scored) codes submitted within the given timeframe and saves them locally for plagiarism check.

Collects all accepted (partial and full scored) codes submitted within the given timeframe of any contest.

ARITRA BELEL 2 Dec 28, 2021
Guesslang detects the programming language of a given source code

Detect the programming language of a source code

Y. SOMDA 618 Dec 29, 2022
Python package to parse and generate C/C++ code as context aware preprocessor.

Devana Devana is a python tool that make it easy to parsing, format, transform and generate C++ (or C) code. This tool uses libclang to parse the code

5 Dec 28, 2022
fixup: Automatically add and remove python import statements

fixup: Automatically add and remove python import statements The goal is that running fixup my_file.py will automatically add or remove import stateme

2 May 08, 2022
Optional static typing for Python 3 and 2 (PEP 484)

Mypy: Optional Static Typing for Python Got a question? Join us on Gitter! We don't have a mailing list; but we are always happy to answer questions o

Python 14.4k Jan 05, 2023
Static type checker for Python

Static type checker for Python Speed Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fas

Microsoft 9.4k Jan 07, 2023
Collection of library stubs for Python, with static types

typeshed About Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages as con

Python 3.3k Jan 02, 2023
Pymwp is a tool for automatically performing static analysis on programs written in C

pymwp: MWP analysis in Python pymwp is a tool for automatically performing static analysis on programs written in C, inspired by "A Flow Calculus of m

Static Analyses of Program Flows: Types and Certificate for Complexity 2 Dec 02, 2022
Learning source code review, spot vulnerability, find some ways how to fix it.

Learn Source Code Review Learning source code review, spot vulnerability, find some ways how to fix it. WordPress Plugin Authenticated Stored XSS on C

Shan 24 Dec 31, 2022
This is a Python program to get the source lines of code (SLOC) count for a given GitHub repository.

This is a Python program to get the source lines of code (SLOC) count for a given GitHub repository.

Nipuna Weerasekara 2 Mar 10, 2022
Code audit tool for python.

Pylama Code audit tool for Python and JavaScript. Pylama wraps these tools: pycodestyle (formerly pep8) © 2012-2013, Florent Xicluna; pydocstyle (form

Kirill Klenov 966 Dec 29, 2022
TidyPy is a tool that encapsulates a number of other static analysis tools and makes it easy to configure, execute, and review their results.

TidyPy Contents Overview Features Usage Docker Configuration Ignoring Issues Included Tools Included Reporters Included Integrations Extending TidyPy

Jason Simeone 33 Nov 27, 2022
A Python utility / library to sort imports.

Read Latest Documentation - Browse GitHub Code Repository isort your imports, so you don't have to. isort is a Python utility / library to sort import

Python Code Quality Authority 5.5k Jan 06, 2023
An interpreter for the X1 bytecode.

X1 Bytecode Interpreter The X1 Bytecode is bytecode designed for simplicity in programming design and compilation. Bytecode Instructions push

Thanasis Tzimas 1 Jan 15, 2022
Typing-toolbox for Python 3 _and_ 2.7 w.r.t. PEP 484.

Welcome to the pytypes project pytypes is a typing toolbox w.r.t. PEP 484 (PEP 526 on the road map, later also 544 if it gets accepted). Its main feat

Stefan Richthofer 188 Dec 29, 2022
An analysis tool for Python that blurs the line between testing and type systems.

CrossHair An analysis tool for Python that blurs the line between testing and type systems. THE LATEST NEWS: Check out the new crosshair cover command

Phillip Schanely 836 Jan 08, 2023
coala provides a unified command-line interface for linting and fixing all your code, regardless of the programming languages you use.

"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." ― John F. Woods coala provides a

coala development group 3.4k Jan 02, 2023