C/C++ Dependency Analyzer: a rewrite of John Lakos' dep_utils (adep/cdep/ldep) from

Overview

logo

https://travis-ci.org/rakhimov/cppdep.svg?branch=master 'Build status' Code Health

cppdep performs dependency analysis among components/packages/package groups of a large C/C++ project. This is a rewrite of dep_utils(adep/cdep/ldep), which is provided by John Lakos' book "Large-Scale C++ Software Design", Addison Wesley (1996).

Limitations

  • Indirect extern declarations of global variables or functions instead of including the proper component header with the declarations.
  • Embedded dynamic dependencies, such as dynamic loading and configurable internal services.
  • Preprocessing or macro expansion is not performed. Dependency inclusion via preprocessor meta-programming is not handled.
  • Dependency exclusion with C style multi-line comments or macros is not respected.

Requirements

  1. Python 2.7 or 3.4+
  2. NetworkX
  3. pydot
  4. pydotplus
  5. PyYAML
  6. PyKwalify 1.6.0+

The dependencies can be installed with pip.

$ sudo pip install -r requirements.txt

Installation

From the source:

$ ./setup.py install

The latest stable release from PyPi:

$ pip install cppdep

Usage

Create a configuration file that describes the project for analysis. config_schema.yml is given for guidance.

In the root directory of the project with the configuration file, run the following command to generate dependency analysis reports and graphs.

$ cppdep -c /path/to/config/file

More documentation and example configurations can be found in project wiki.

Acknowledgments

  • John Lakos for inventing the analysis and providing dep_utils.
  • Zhichang Yu for rewriting dep_utils into Python.
Comments
  • Behavior specification for anomalous conflicting component files

    Behavior specification for anomalous conflicting component files

    Anomalous (rare, error) case of having "component.c" and "component.cc" or "component.h" and "component.hpp" at the same time trying to define the same component. In other words, files with the same basenames but different extensions in header and/or source groups.

    opened by rakhimov 1
  • Missing 'pydot' dependency fails on networkx nx_pydot

    Missing 'pydot' dependency fails on networkx nx_pydot

    Traceback (most recent call last):
      File "/.../python3env/bin/cppdep", line 11, in <module>
        sys.exit(main())
      File "/.../python3env/lib/python3.5/site-packages/cppdep/__main__.py", line 59, in main
        analysis.analyze(printer, args)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 797, in analyze
        lambda x: isinstance(x, PackageGroup)))
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 779, in _analyze
        digraph.write_dot(graph_name)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/graph.py", line 273, in write_dot
        write_dot(self.digraph, file_basename + '.dot')
      File "<decorator-gen-416>", line 2, in write_dot
      File "/.../python3env/lib/python3.5/site-packages/networkx/utils/decorators.py", line 224, in _open_file
        result = func(*new_args, **kwargs)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 54, in write_dot
        P = to_pydot(G)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 199, in to_pydot
        pydot = _import_pydot()
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 348, in _import_pydot
        import pydot
    
    bug 
    opened by rakhimov 0
  • Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    If component/project names contain paths as identifiers, the path separator on Windows \ followed by 'U' or 'u' is interpreted as a Unicode literal by pydotplus plotting on Windows with python 2.7.

    This problem does not show up on python 3.

    For the sake of consistency in the report (stable report) and testing, use Unix path separator in Ids.

    bug 
    opened by rakhimov 0
  • Handle 'ipp' template implementation/source files

    Handle 'ipp' template implementation/source files

    Even though templates are only in headers, ipp files could be considered as implementation pair of the interface header. This occurs, for example, in Boost. #29.

    opened by rakhimov 0
  • Redundant 'ldep' printing of cumulative dependencies

    Redundant 'ldep' printing of cumulative dependencies

    Unlike the original ldep, the cppdep is printing the whole cumulative dependencies; that is, the all link time dependencies are printed. This information is visually provided in the graph, and implicitly provided in the original ldep '-l | -L' flags.

    It seems like the current implementation of the cumulative dependency printing is geared towards debugging rather than analysis report.

    Fix this printing by implementing the original behavior of ldep.

    bug 
    opened by rakhimov 0
  • Pairing header and implementation files in different locations

    Pairing header and implementation files in different locations

    Header and implementation files can be located in different directories, e.g., headers in include and implementation in src. The current implementation assumes the same location.

    bug enhancement 
    opened by rakhimov 0
  • Deduce external packages from the include directives w/o filesystem search

    Deduce external packages from the include directives w/o filesystem search

    The package can be deduced from the include directives following the pattern: "<package/header>", e.g., "<boost/any.hpp>". This approach will avoid the relatively expensive lookup of the header from the system. Whether such external header actually exists on the system or not is irrelevant to the dependency analysis.

    This heuristic would make the configuration simpler and more robust for cross-platform work.

    In addition, the standard library headers can be "hard-coded" into the script since these are the most likely to be used/searched by the source files.

    There's a small chance of false positives if the project under analysis happens not to conform to this convention and invents its own packages with the same names as external ones, e.g., boost, qt, libxml.

    enhancement 
    opened by rakhimov 0
  • Double counting of common components in CCD

    Double counting of common components in CCD

    The current implementation overcounts link time dependencies with common components deep in the dependency graph. Consider: A->B->C, A->D->C. CCD(A) must be 4 (4 object files in total.) The current approach overcounts the common object C, so CCD(A) ends up being 5.

    bug 
    opened by rakhimov 0
  • Extended definition for 'Component'

    Extended definition for 'Component'

    Even though John Lakos defined a component as a pair of h and c files, C++ can have template only components residing only in header files (e.g., STL/Boost/etc.). Moreover, some header-only components may contain only inline functions or macros without any need for an implmentation file (e.g., inline math, Boost PPL). For these reason, unpaired header files can be counted as components by default.

    In addition, the implementation file containing the main function of an application could be considered as a component as well (an entry point).

    bug enhancement 
    opened by rakhimov 0
  • Incorrect dependency processing with file basenames

    Incorrect dependency processing with file basenames

    The current implementation uses the basename of the included files as the key in dependency search. This approach ignores the current location of the dependent files and the language include rules, i.e. <header.h> vs. "header.h". Moreover, it results in name conflicts for files that may belong to completely separate projects or packages.

    bug 
    opened by rakhimov 0
  • Compatibility fixes

    Compatibility fixes

    This PR contains some fixes that I needed to get the cppdep tool running in my recent python 3.8 environment. Also handling of errors in the yaml file was adapted a little bit to get more helpful error messages

    opened by jsinge 0
  • Missing graph module

    Missing graph module

    I've just installed this project, tried to run it but I get this error message.

    Traceback (most recent call last):
      File "cppdep.py", line 36, in <module>
        from .graph import Graph
    ModuleNotFoundError: No module named '__main__.graph'; '__main__' is not a package
    
    bug help wanted question 
    opened by nbourre 2
  • Python 3.6 os.path.commonprefix expects List

    Python 3.6 os.path.commonprefix expects List

      File "/.../cppdep/cppdep.py", line 92, in path_common
        path = os.path.commonprefix(paths)
      File "/home/olzhas/temp/pyenv/lib/python3.6/genericpath.py", line 76, in commonprefix
        if not isinstance(m[0], (list, tuple)):
    TypeError: 'set' object does not support indexing
    
    bug 
    opened by rakhimov 1
  • Include Wrangler Analysis

    Include Wrangler Analysis

    Incorporation of include-wrangler analysis

    • header file cost
    • include cost
    • translation unit cost
    • other features

    In collaboration with include-wrangler author @lukedodd

    enhancement 
    opened by rakhimov 0
  • Options to disable/enable warnings

    Options to disable/enable warnings

    • [ ] Duplicate include
    • [ ] Redundant (transitive from the component header) include
    • [ ] Missing a component header (incomplete component)
    • [ ] Failure to locate a header from an include directive
    enhancement 
    opened by rakhimov 0
Releases(0.2.0)
  • 0.2.0(Feb 3, 2017)

    Added

    • Pairing header and implementation files in different locations (#19)
    • Handle 'ipp' template implementation source files (#31)
    • Behavior specification for anomalous conflicting component files (#27)
    • Implement ignore/exclude paths (#23)
    • Accept glob pattern for source paths (#36)
    • Project wiki pages
    • Regex pattern based include directive classification (#22)
    • Deduce external packages from the include directive w/o filesystem search (#18)
    • Handle header files w/o extensions (Boost/STL/Qt/etc.) (#32)
    • Use POSIX path separator in component names (for cross-platform report stability)
    • Configuration file validation against the schema (with PyKwalify)

    Changed

    • pytest instead of nose
    • YAML configuration files instead of XML (#24)

    Removed

    • Implicit single-path alias Package construction

    Fixed

    • Exception leaks out of main()
    • Unicode Escape Error on graph dot on Windows with Python 2.7 (#35)
    • Python3 UnicodeDecodeError for 'utf-8' in source files (#30)
    • Logging: Type Error: not all arguments converted during string formatting (#28)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 6, 2017)

    Added

    • The original ldep '-l|-L' options to print dependencies (#20)
    • '-o' to print reports into a file
    • Warn about duplicate and redundant includes (#13)
    • Extended definition for 'Component' (#7)
    • PEP-257 conformance (#2)
    • PEP-8 conformance (#1)
    • Python 3 support
    • PyPI package
    • XML configuration example and RNG schema
    • Travis CI (Linux, OS X) and AppVeyor CI (Windows) setups

    Changed

    • Differentiate 'paths' into source, include, and alias.
    • Print warnings to stderr instead of stdout (#12)
    • Report Component levels instead of Graph layers (#9)
    • Refactor the procedural design into the object-oriented design (#4)
    • Change '-f' flag into '-c' flag
    • Replace optparse with argparse
    • XML configuration file format

    Removed

    • Redundant printing a list of cumulative dependencies (#20)
    • Indirect missing-header include warnings
    • Global cross-package and cross-package-group component dependency analysis
    • 'details-of-components/--debug' verbosity
    • dot2any.py helper script
    • Manual profiling code (use pyvmmonitor instead)
    • Manual testing code (automated with nosetest)

    Fixed

    • Level 0 External components missing from the report and graph (#21)
    • Incorrect dependency processing with file basenames (#6)
    • Wrong level calculation for cycles (#8)
    • Double counting of common components in CCD calculations (#11)
    • Missing cycles from the Dot graph (#10)
    • Outdated networkx API usage
    Source code(tar.gz)
    Source code(zip)
Owner
Olzhas Rakhimov
Olzhas Rakhimov
A simple stopwatch for measuring code performance with static typing.

A simple stopwatch for measuring code performance. This is a fork from python-stopwatch, which adds static typing and a few other things.

Rafael 2 Feb 18, 2022
Find usage statistics (imports, function calls, attribute access) for Python code-bases

Python Library stats This is a small library that allows you to query some useful statistics for Python code-bases. We currently report library import

Francisco Massa 13 May 02, 2022
pycallgraph is a Python module that creates call graphs for Python programs.

Project Abandoned Many apologies. I've stopped maintaining this project due to personal time constraints. Blog post with more information. I'm happy t

gak 1.7k Jan 01, 2023
Code audit tool for python.

Pylama Code audit tool for Python and JavaScript. Pylama wraps these tools: pycodestyle (formerly pep8) © 2012-2013, Florent Xicluna; pydocstyle (form

Kirill Klenov 966 Dec 29, 2022
Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

Alex Grönholm 1.1k Dec 19, 2022
Typing-toolbox for Python 3 _and_ 2.7 w.r.t. PEP 484.

Welcome to the pytypes project pytypes is a typing toolbox w.r.t. PEP 484 (PEP 526 on the road map, later also 544 if it gets accepted). Its main feat

Stefan Richthofer 188 Dec 29, 2022
Collection of library stubs for Python, with static types

typeshed About Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages as con

Python 3.3k Jan 02, 2023
Find dead Python code

Vulture - Find dead code Vulture finds unused code in Python programs. This is useful for cleaning up and finding errors in large code bases. If you r

Jendrik Seipp 2.4k Jan 03, 2023
An interpreter for the X1 bytecode.

X1 Bytecode Interpreter The X1 Bytecode is bytecode designed for simplicity in programming design and compilation. Bytecode Instructions push

Thanasis Tzimas 1 Jan 15, 2022
ticktock is a minimalist library to profile Python code

ticktock is a minimalist library to profile Python code: it periodically displays timing of running code.

Victor Benichoux 30 Sep 28, 2022
Static type checker for Python

Static type checker for Python Speed Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fas

Microsoft 9.4k Jan 07, 2023
A system for Python that generates static type annotations by collecting runtime types

MonkeyType MonkeyType collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type

Instagram 4.1k Jan 02, 2023
coala provides a unified command-line interface for linting and fixing all your code, regardless of the programming languages you use.

"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." ― John F. Woods coala provides a

coala development group 3.4k Jan 02, 2023
TidyPy is a tool that encapsulates a number of other static analysis tools and makes it easy to configure, execute, and review their results.

TidyPy Contents Overview Features Usage Docker Configuration Ignoring Issues Included Tools Included Reporters Included Integrations Extending TidyPy

Jason Simeone 33 Nov 27, 2022
Print a directory tree structure in your Python code.

directory-structure Print a directory tree structure in your Python code. Download You can simply: pip install directory-structure Or you can also: Cl

Gabriel Stork 45 Dec 19, 2022
This is a Python program to get the source lines of code (SLOC) count for a given GitHub repository.

This is a Python program to get the source lines of code (SLOC) count for a given GitHub repository.

Nipuna Weerasekara 2 Mar 10, 2022
The strictest and most opinionated python linter ever!

wemake-python-styleguide Welcome to the strictest and most opinionated python linter ever. wemake-python-styleguide is actually a flake8 plugin with s

wemake.services 2.1k Jan 05, 2023
Collects all accepted (partial and full scored) codes submitted within the given timeframe and saves them locally for plagiarism check.

Collects all accepted (partial and full scored) codes submitted within the given timeframe of any contest.

ARITRA BELEL 2 Dec 28, 2021
Typical: Fast, simple, & correct data-validation using Python 3 typing.

typical: Python's Typing Toolkit Introduction Typical is a library devoted to runtime analysis, inference, validation, and enforcement of Python types

Sean 170 Dec 26, 2022
Performant type-checking for python.

Pyre is a performant type checker for Python compliant with PEP 484. Pyre can analyze codebases with millions of lines of code incrementally – providi

Facebook 6.2k Jan 07, 2023