Pipeline is an asset packaging library for Django.

Overview

Pipeline

Jazzband GitHub Actions Coverage Documentation Status

Pipeline is an asset packaging library for Django, providing both CSS and JavaScript concatenation and compression, built-in JavaScript template support, and optional data-URI image and font embedding.

Installation

To install it, simply:

pip install django-pipeline

Documentation

For documentation, usage, and examples, see : https://django-pipeline.readthedocs.io

Comments
  • Load times have increased dramatically since upgrade to 1.4.x

    Load times have increased dramatically since upgrade to 1.4.x

    Hello,

    We just tried upgrading django-pipeline from version 1.3.25 to 1.4.x (multiple minor versions tested, including latest: 1.4.6). Every request rendering static files went from 0.5 sec to about 20 secs to render.

    We tried to debug what led to this increase but could not find it. After downgrading to 1.3 the load times were again at 0.5 secs per page.

    opened by bobvoorneveld 44
  • When less file exists in a different folder depth than output, staticfiles dies looking for images

    When less file exists in a different folder depth than output, staticfiles dies looking for images

    I could swear this worked in 1.1.27...

    When any of my less files specify background: url(''), the url is relative assuming that all compressed CSS will end up in css/*.css, and therefore the image urls being with '../img/'.

    However, it seems that pipeline has passed paths off to django staticfiles that are all relative to their original less files, which leads to the image not being found:

    Traceback (most recent call last):
    ...
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
        collected = self.collect()
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/management/commands/collectstatic.py", line 120, in collect
        for original_path, processed_path, processed in processor:
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 270, in post_process
        content = pattern.sub(converter, content)
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 213, in converter
        hashed_url = self.url(unquote(joined_result), force=True)
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 160, in url
        hashed_name = self.hashed_name(clean_name).replace('\\', '/')
      File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 123, in hashed_name
        (clean_name, self))
    ValueError: The file 'less/img/wilford-brimley-diabeetus-cat.jpg' could not be found with <simple_pipeline.myapp.pipeline_storage.S3PipelineStorage object at 0x1027f2a90>.
    

    No to go figure out if/where this ever worked...

    [1] http://pastebin.com/MGtcj9rA (full stacktrace here)

    question 
    opened by estebistec 27
  • Do not create compiled JS and CSS in source directory

    Do not create compiled JS and CSS in source directory

    If I have a file, 'foo.coffee', when I reference that file through the tag, it creates a file next to it called 'foo.js'. The problem with this is it makes it very difficult to have a version control system ignore these files. It is very unlikely that I would want to blanket ignore .css or .js files, so that isn't an option. Instead, I'm left ignoring files individually when I add a new less or coffee file.

    It would be nice if pipeline placed the files in their own directory, so I could blanket ignore that directory. As an example, if I have '{static}/foo/bar.coffee', pipeline could place the compiled file in '{static}/{pipelined}/foo/bar.js', where "pipelined" could be a configurable setting.

    opened by SoftwareMaven 21
  • FinderStorage don't write where it should

    FinderStorage don't write where it should

    https://github.com/cyberdelia/django-pipeline/blob/master/pipeline/storage.py#L89 <- bug here

    path is string and os.path.dirname(name) is string too.

    Searching substring in this case is so confusing - need more complicated logic.

    bug 
    opened by dreadatour 21
  • Less compiler not respecting relative paths with @import

    Less compiler not respecting relative paths with @import

    I'm running django-pipeline version 1.1.22 (latest), and the following isn't working yet in .less files:

     @import "bootstrap-1.4/bootstrap.less";
    

    The reason one would do this would be to include the Less library and be able to predefine variables and be able to use the mixins exposed by the library in one's own Less code. For instance:

    @basefont: 14px; /* in bootstrap, it's 13 by default, so we override it before it's imported */
    
    @import "bootstrap-1.4/bootstrap.less";
    
    .ghostly { /* we use bootstrap's defined .opacity mixin to create this effect. */
        .opacity(90);
    }
    

    Currently, this doesn't work and yields:

    Environment:
    
    
    Request Method: GET
    Request URL: http://localhost:8000/
    
    Django Version: 1.3
    Python Version: 2.7.1
    Installed Applications:
    ['django.contrib.auth',
     'django.contrib.contenttypes',
     'django.contrib.sessions',
     'django.contrib.sites',
     'django.contrib.messages',
     'django.contrib.admin',
     'django.contrib.admindocs',
     'django.contrib.staticfiles',
     'django_bcrypt',
     'pipeline',
     'reversion',
     'south',
     'myproject.web']
    Installed Middleware:
    ('django.middleware.common.CommonMiddleware',
     'django.contrib.sessions.middleware.SessionMiddleware',
     'django.middleware.csrf.CsrfViewMiddleware',
     'django.contrib.auth.middleware.AuthenticationMiddleware',
     'django.contrib.messages.middleware.MessageMiddleware')
    
    
    Template error:
    In template /home/rfkrocktk/Desktop/myproject/src/myproject/web/templates/desktop/root.html, error at line 6
       Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
        from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
        from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
        from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
        from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
        from /usr/local/bin/lessc:19:in `load'
        from /usr/local/bin/lessc:19
    
       1 : {% load static %}
    
    
       2 : {% load compressed %}
    
    
       3 : <!DOCTYPE html>
    
    
       4 : <html>
    
    
       5 :  <head>
    
    
       6 :      {% block stylesheets %} {% compressed_css 'all' %} {% endblock %}
    
    
       7 :      <title>{% block title %}{% endblock %}&nbsp;&raquo;&nbsp;myproject</title>
    
    
       8 :  </head>
    
    
       9 :  <body>
    
    
       10 :         {% block body %}{% endblock %}
    
    
       11 :         {% block scripts %}{% compressed_js 'all' %}{% endblock %}
    
    
       12 :     </body>
    
    
       13 : </html>
    
    
       14 : 
    
    Traceback:
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/handlers/base.py" in get_response
      111.                         response = callback(request, *callback_args, **callback_kwargs)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/views/generic/simple.py" in direct_to_template
      28.     return HttpResponse(t.render(c), mimetype=mimetype)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      123.             return self._render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
      117.         return self.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
      127.         return compiled_parent._render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
      117.         return self.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
      127.         return compiled_parent._render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
      117.         return self.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
      64.             result = block.nodelist.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
      744.                 bits.append(self.render_node(node, context))
    File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
      73.             result = node.render(context)
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/templatetags/compressed.py" in render
      30.             package['paths'] = self.packager.compile(package['paths'])
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/packager.py" in compile
      50.         return self.compiler.compile(paths)
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in compile
      25.                         compiled_content = compiler.compile_file(content, storage.path(path))
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/less/__init__.py" in compile_file
      17.         content = self.execute_command(command)
    File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in execute_command
      88.             raise CompilerError(error)
    
    Exception Type: TemplateSyntaxError at /
    Exception Value: Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
        from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
        from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
        from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
        from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
        from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
        from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
        from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
        from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
        from /usr/local/bin/lessc:19:in `load'
        from /usr/local/bin/lessc:19
    
    bug awaiting response 
    opened by naftulikay 17
  • Compiler @import don't work

    Compiler @import don't work

    With the below situation, our 'base' bundle is still being compressed and written to a file called 'css/base.css' (where it should be 'css/thatguyismystyle.base.min.css' )

    The template tag still writes out the desired filename (which is 'css/thatguyismystyle.base.min.css' )


    Django==1.3
    django-pipeline==1.2.5
    django-staticfiles==1.2.1
    

        from os.path import join
        from settings import SERVER_ROOT
        from collections import OrderedDict as OD
    
        NODE_BIN = join(SERVER_ROOT, 'node_modules/.bin')
    
        #
        # Pipelined CSS Files
        #
    
    
        PIPELINE_CSS = OD((
            ('base', {
                'source_filenames': (
                    'css/base.less',
                ),
    
                'output_filename':
                    'css/thatguyismystyle.base.min.css',
    
                'extra_context': {
                    'media': 'screen',
                },
            }),
            ('theme', {
                'source_filenames': (
                    'css/theme/base.less',
                ),
                'output_filename':
                    'css/theme/thatguyismystyle.theme.min.css',
    
                'extra_context': {
                    'media': 'screen',
                },
            }),
            ('theme-light', {
                'source_filenames': (
                    'themes/light/css/base.less'
                ),
                'output_filename':
                    'themes/thatguyismystyle.colour.light.min.css',
                'extra_context': {
                    "media": "screen",
                }
            }),
            ('theme-dark', {
                'source_filenames': (
                    'themes/dark/css/base.less'
                ),
                'output_filename':
                    'themes/thatguyismystyle.colour.dark.min.css',
                'extra_context': {
                    "media": "screen",
                }
            }),
        ))
    
        #
        # Pipelined JS FILES
        #
    
        BASE_JS_FRAMEWORK = (
            "js/lib/modernizr.min.js",
            "js/lib/jquery.min.1.7.1.js",
            "js/lib/history.js/jquery.history.js",
            "js/lib/moment.min.pretty.js",
            "js/lib/moment.strftime.min.js",
        )
        BOOTSTRAP_JS = (
            'js/lib/bootstrap/bootstrap-alert.js',
            'js/lib/bootstrap/bootstrap-button.js',
            'js/lib/bootstrap/bootstrap-carousel.js',
            'js/lib/bootstrap/bootstrap-collapse.js',
            'js/lib/bootstrap/bootstrap-dropdown.js',
            'js/lib/bootstrap/bootstrap-modal.js',
            'js/lib/bootstrap/bootstrap-popover.js',
            'js/lib/bootstrap/bootstrap-scrollspy.js',
            'js/lib/bootstrap/bootstrap-tab.js',
            'js/lib/bootstrap/bootstrap-tooltip.js',
            'js/lib/bootstrap/bootstrap-transition.js',
            'js/lib/bootstrap-contrib/*.js',
        )
        CKEDITOR_JS = (
            'js/lib/ckeditor/*.js',
        )
        TINYMCS_JS = (
            'js/lib/tinymce/tiny_mce.js',
            'js/lib/tinymce/langs/*.js',
            'js/lib/tinymce/jquery.tinymce.js',
        )
    
    
        PIPELINE_JS = OD((
            ('backend', {
                'source_filenames': (
                    'js/lib/jquery.js',
                ) + BOOTSTRAP_JS
                  + CKEDITOR_JS + (
                    'js/lib/jquery.*.js',
                    'js/lib/thatguyismystyle.*.js',
                    'js/application.js',
                ),
                'output_filename': 'js/.js',
            }),
            ('profile-pages', {
                'source_filenames': (
                ),
                'output_filename': 'js/profile.js'
            })
        ))
        PIPELINE_JS = dict()
    
        PIPELINE_DISABLE_WRAPPER = True
    
        #
        # Compilers
        #
    
        PIPELINE_STORAGE = 'staticfiles.storage.StaticFilesStorage'
        STATICFILES_STORAGE = 'staticfiles.storage.StaticFilesStorage'
        PIPELINE=True
    
        PIPELINE_COMPILERS = (
            'pipeline.compilers.less.LessCompiler',
            'pipeline.compilers.coffee.CoffeeScriptCompiler',
        )
        PIPELINE_LESS_BINARY = join( NODE_BIN, 'lessc' )
    
        #
        # Compressors
        #
        PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.jsmin.JSMinCompressor'
        PIPELINE_CSS_COMPRESSOR = 'pipeline.compressors.cssmin.CssminCompressor'
    
    
        STATICFILES_FINDERS = (
         'staticfiles.finders.AppDirectoriesFinder',
        )
    
        STATICFILES_IGNORE_PATTERNS = (
            "*.txt",
            "tests",
            "css/*.old",
            "/opt/webfiles/common/*.txt",
            "/opt/webfiles/common/temp",
        )
    
    bug 
    opened by airtonix 15
  • Ignore missing files

    Ignore missing files

    In large projects that use dozens of third-party libraries, invariably, some library will reference a url(../../..somefile) which doesn't exist. When running manage.py collectstatic with django-pipeline, this will result in an incredibly unhelpful error like:

    ValueError: The file 'tinymce/js/tinymce/plugins/compat3x/img/icons.gif' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x1ca6750>.
    

    Not only does pipeline not mention which file attempts to include this file, searching for img/icons.gif myself results in 42 matches. Even if I could locate the issue, I can't easily fix the missing file, since it's a third-party library. This means pipeline is effectively blocking all my deployments because of a missing file I don't even use.

    Ideally, pipeline should report which resource is referencing a missing file and allow specifying an option to ignore it. This is very similar to the https://github.com/adamcharnock/django-pipeline-forgiving fork, although I think this would be a good idea for the main branch.

    opened by chrisspen 14
  • 1.4.2 requires absolute paths support from static storage

    1.4.2 requires absolute paths support from static storage

    I'm using the following static storage:

    class S3PipelineStorage(PipelineMixin, S3BotoStorage):
        """S3 storage with pipeline magic"""
        pass
    

    Which is almost the same as the one from pipeline docs (even simpler because it doesnt use CachedFilesMixin) . It's fine with 1.3.x but breaks with 1.4.2 because it tries to run sass compiler using absolute path from s3. S3BotoStorage doesn't support absolute paths and throws this long exception:

    /home/bender/myapps/foo/manage.py in <module>()
          8     from django.core.management import execute_from_command_line
          9 
    ---> 10     execute_from_command_line(sys.argv)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/__init__.pyc in execute_from_command_line(argv)
        383     """
        384     utility = ManagementUtility(argv)
    --> 385     utility.execute()
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/__init__.pyc in execute(self)
        375             sys.stdout.write(self.main_help_text() + '\n')
        376         else:
    --> 377             self.fetch_command(subcommand).run_from_argv(self.argv)
        378 
        379 
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in run_from_argv(self, argv)
        286         handle_default_options(options)
        287         try:
    --> 288             self.execute(*args, **options.__dict__)
        289         except Exception as e:
        290             if options.traceback or not isinstance(e, CommandError):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in execute(self, *args, **options)
        336                     not options.get('skip_checks')):
        337                 self.check()
    --> 338             output = self.handle(*args, **options)
        339             if output:
        340                 if self.output_transaction:
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in handle(self, *args, **options)
        531         if args:
        532             raise CommandError("Command doesn't accept any arguments")
    --> 533         return self.handle_noargs(**options)
        534 
        535     def handle_noargs(self, **options):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.pyc in handle_noargs(self, **options)
        166             raise CommandError("Collecting static files cancelled.")
        167 
    --> 168         collected = self.collect()
        169         modified_count = len(collected['modified'])
        170         unmodified_count = len(collected['unmodified'])
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.pyc in collect(self)
        112             processor = self.storage.post_process(found_files,
        113                                                   dry_run=self.dry_run)
    --> 114             for original_path, processed_path, processed in processor:
        115                 if isinstance(processed, Exception):
        116                     self.stderr.write("Post-processing '%s' failed!" % original_path)
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/storage.pyc in post_process(self, paths, dry_run, **options)
         28         from pipeline.packager import Packager
         29         packager = Packager(storage=self)
    ---> 30         for package_name in packager.packages['css']:
         31             package = packager.package_for('css', package_name)
         32             output_file = package.output_filename
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in pack_stylesheets(self, package, **kwargs)
         94         return self.pack(package, self.compressor.compress_css, css_compressed,
         95                          output_filename=package.output_filename,
    ---> 96                          variant=package.variant, **kwargs)
         97 
         98     def compile(self, paths, force=False):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in pack(self, package, compress, signal, **kwargs)
        103         if self.verbose:
        104             print("Saving: %s" % output_filename)
    --> 105         paths = self.compile(package.paths, force=True)
        106         content = compress(paths, **kwargs)
        107         self.save_file(output_filename, content)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in compile(self, paths, force)
         97 
         98     def compile(self, paths, force=False):
    ---> 99         return self.compiler.compile(paths, force=force)
        100 
        101     def pack(self, package, compress, signal, **kwargs):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/compilers/__init__.pyc in compile(self, paths, force)
         54         else:
         55             with futures.ThreadPoolExecutor(max_workers=multiprocessing.cpu_count()) as executor:
    ---> 56                 return list(executor.map(_compile, paths))
         57 
         58     def output_path(self, path, extension):
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in map(self, fn, *iterables, **kwargs)
        578             for future in fs:
        579                 if timeout is None:
    --> 580                     yield future.result()
        581                 else:
        582                     yield future.result(end_time - time.time())
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in result(self, timeout)
        398                 raise CancelledError()
        399             elif self._state == FINISHED:
    --> 400                 return self.__get_result()
        401 
        402             self._condition.wait(timeout)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in __get_result(self)
        357     def __get_result(self):
        358         if self._exception:
    --> 359             reraise(self._exception, self._traceback)
        360         else:
        361             return self._result
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_compat.pyc in reraise(exc, traceback)
        105     def reraise(exc, traceback):
        106         locals_ = {'exc_type': type(exc), 'exc_value': exc, 'traceback': traceback}
    --> 107         exec('raise exc_type, exc_value, traceback', {}, locals_)
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/thread.pyc in run(self)
         59 
         60         try:
    ---> 61             result = self.fn(*self.args, **self.kwargs)
         62         except BaseException:
         63             e, tb = sys.exc_info()[1:]
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/compilers/__init__.pyc in _compile(input_path)
         34                 if compiler.match_file(input_path):
         35                     output_path = self.output_path(input_path, compiler.output_extension)
    ---> 36                     infile = self.storage.path(input_path)
         37                     outfile = self.output_path(infile, compiler.output_extension)
         38                     outdated = compiler.is_outdated(input_path, output_path)
    
    /home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/files/storage.pyc in path(self, name)
         85         accessed using open() should *not* implement this method.
         86         """
    ---> 87         raise NotImplementedError("This backend doesn't support absolute paths.")
         88 
         89     # The following methods form the public API for storage systems, but with
    
    NotImplementedError: This backend doesn't support absolute paths
    
    opened by rjagielski 12
  • pipeline.exceptions.CompressorError

    pipeline.exceptions.CompressorError

    I occasionally (once every 10 runs or so?) get this error when django-pipeline is compressing my JS:

    Traceback (most recent call last):
      File "manage.py", line 10, in <module>
        execute_from_command_line(sys.argv)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
        utility.execute()
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
        output = self.handle(*args, **options)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
        return self.handle_noargs(**options)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 173, in handle_noargs
        collected = self.collect()
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 118, in collect
        for original_path, processed_path, processed in processor:
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/storage.py", line 39, in post_process
        packager.pack_javascripts(package)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/packager.py", line 110, in pack_javascripts
        return self.pack(package, self.compressor.compress_js, js_compressed, templates=package.templates, **kwargs)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/packager.py", line 104, in pack
        content = compress(paths, **kwargs)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 65, in compress_js
        js = getattr(compressor(verbose=self.verbose), 'compress_js')(js)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/yuglify.py", line 13, in compress_js
        return self.compress_common(js, 'js', settings.PIPELINE_YUGLIFY_JS_ARGUMENTS)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/yuglify.py", line 10, in compress_common
        return self.execute_command(command, content)
      File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 237, in execute_command
        raise CompressorError(stderr)
    pipeline.exceptions.CompressorError: Killed
    

    The only way I have found to continue is to restart the vagrant box that I am developing on.

    opened by ErikEvenson 12
  • AttributeError: 'CachedFileFinder' object has no attribute 'storages'

    AttributeError: 'CachedFileFinder' object has no attribute 'storages'

    I'm having the following problem. Not sure if it's a misconfiguration on my end or a bug.

    I'm using Bower to install some components. I've added to STATICFILES_DIRS the directory where the components are installed.

    Here are some relevant parts from settings.py

    STATICFILES_FINDERS = (
        'pipeline.finders.AppDirectoriesFinder',
        'pipeline.finders.FileSystemFinder',
        'pipeline.finders.PipelineFinder',
        'pipeline.finders.CachedFileFinder',
    )
    STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
    DEBUG = False
    

    When I run collectstatic, I get the following error:

    AttributeError: 'CachedFileFinder' object has no attribute 'storages'
    

    However, If I remove CachedFileFinder, collectstatic works fine.

    opened by honi 12
  • What should be happening?

    What should be happening?

    I've begun implementing Pipeline in our app. I've added the middleware, items to the requirements.txt file, the compressor variable, and the PIPELINE_JS object.

    PIPELINE_JS = {
        'activity': {
            'source_filenames': (
                'assets/js/libs/src/core.libs.js',
                'assets/js/libs/src/datepicker.js',
            ),
            'output_filename': 'assets/js/libs/activity.js',
        }
    }
    

    I've included the template tag on the page, and output the tag call like so:

    {% compressed_js 'activity' %}

    The page loads without error, but nothing appears to be happening. I wanted to make sure something should be happening at this point. I don't see any outputted files, and viewing the rendered source of the page shows an empty line where the call to the template was.

    So what should I expect is happening?

    Also, as a side note, it would be AWESOME if you could include a super simple app using pipeline as a demo. Something that users could download and run locally to see what happens.

    Looking forward to your response.

    opened by commadelimited 12
  • Add Django 4.1 support

    Add Django 4.1 support

    Only a small change was required in the tests to handle this change about removing the type="text/css" attributes for CSS link tags:

    2022-09-29_12-14


    It might make sense when releasing a new version to drop support for Python 3.6 and Django 2.2, 3.0 and 3.1 since they have all reached end of life. If you want I can push that in this PR or seperately

    opened by kevinmarsh 3
  • swapped the project_infile path with the infile path on the compile_f…

    swapped the project_infile path with the infile path on the compile_f…

    collectstatic static fails with the project_infile variable in the compile_file method because the finder looks for static files in the static directories not from static_root.

    opened by OzzyGrammer 0
  • [Question][Bug] Infile paramater value wrong after updating from 1.6.14 to 2.0.7

    [Question][Bug] Infile paramater value wrong after updating from 1.6.14 to 2.0.7

    We had to upgrade django-pipeline from 1.6.14 to at least 2.0.7 since we are currently in the process of upgrading django to version 3.2. After we made the update, our static files were not behaving the same anymore (css not loading). After a few days of investigating I saw that the infile parameter that is passed into the compile_file function in the LessCompiler class were not the same as the value we usually got before we made the update.

    Seems like the infile value comes from compile function on the Compiler class in the init file.

        def compile(self, paths, compiler_options={}, force=False):
            def _compile(input_path):
                for compiler in self.compilers:
                    compiler = compiler(verbose=self.verbose, storage=self.storage)
                    if compiler.match_file(input_path):
                        try:
                            infile = self.storage.path(input_path)
                        except NotImplementedError:
                            infile = finders.find(input_path)
                        project_infile = finders.find(input_path)
                        outfile = compiler.output_path(infile, compiler.output_extension)
                        outdated = compiler.is_outdated(project_infile, outfile)
                        compiler.compile_file(project_infile, outfile,
                                              outdated=outdated, force=force,
                                              **compiler_options)
    
                        return compiler.output_path(input_path, compiler.output_extension)
                else:
                    return input_path
    

    I see the infile value is never passed into the compile_file function. No matter what happens in the try except, another variable called project_infile is created underneath and passed to compile_file, which in our case has the wrong value. Infile has the correct value. On the older version that we used the project_infile didnt exist and the infile value was passed to compile_file. Thank you in advance.

    opened by Willem-Nieuwoudt 0
  • [Feature Request]: self.request to context in render_js

    [Feature Request]: self.request to context in render_js

    This is a feature request to add self.request to context in render_js on JavascriptNode.

    This would be handy for cases where request context variables need to be rendered in the js template, for example, with django-csp you would be able to add a nonce in a custom template without having to extend JavascriptNode and writing a new tag to use it.

    My suggestion is to add a key to extra_context which you can declare in settings with a boolean value that defaults to false. If true it adds self.request to context which will then be usable in the template.

    If it's something anybody else is interested in, I could create a pull request for it.

    opened by mfvictor 0
  • pipeline compression skips over js when using yuglify

    pipeline compression skips over js when using yuglify

    Hi there,

    I am a bit new to using js. I am having a strange output, where the functions inside a js file are being skipped over during compression using yuglify. I am getting no errors, except when checking the output the functions are not present!

    The file which is being ignored

    // application.js
    function changeValueProp() {
        if (document.getElementById('ShowHideProp').value == 'Show Properties'){
          document.getElementById('ShowHideProp').value = 'Hide Properties';
        }
        else{
          document.getElementById('ShowHideProp').value = 'Show Properties';
        }
      }
    

    In my pipeline config I am using

    {
        "JAVASCRIPT": {
         "application": {"source_filenames":  (f'{APP_STATIC_DIR}/application.js',f'{APP_STATIC_DIR}/application01.js',),
                                 "output_filename":    'compressed/js/application.js'},
                              },
        "YUGLIFY_BINARY": f"{BASE}/node_modules/yuglify/bin/yuglify",
    }
    

    I just checked that if I use "DISABLE_WRAPPER":True, the function from application.js is compressed and put in the output file. In ReadTheDocs, the DISABLE_WRAPPER:False is recommended for improving performance.

    Is it possible to use the wrapper without it skipping over the js described above ?

    BR

    SK

    opened by esskaey 0
Releases(2.0.8)
  • 2.0.8(Jan 10, 2022)

    • Added Django 4.0 compatibility. Thanks to @kevinmarsh (#760)
    • Add tests for Django 4.0, Python 3.9 and Python 3.10. Thank to @kevinmarsh (#739)
    • Introduce CODE_OF_CONDUCT.md for the project. Thank to @hugovk (#758)
    • Add precision in the documentation for PipelineCachedStorage. Thank to @gatsinski (#739)
    • Drop support for slimit compressor (#765) due to package not released an official version for Python 3 and not any new package release from 2013.
    • Edit github actions matrix: django 3.2.9 support python 3.10, remove python 4.0 (doesn't exist) and exclude pypy-3.8 for django-main.
    • Add .pre-commit-config.yaml. Thanks to @hugovk (#762)
    • Update package.json due to CoffeeScript on NPM has moved to "coffeescript"
    • Update setup.py with Django 4.0 and Python 3.10
    Source code(tar.gz)
    Source code(zip)
  • 2.0.7(Oct 21, 2021)

  • 2.0.6(Jan 18, 2021)

  • 2.0.5(Aug 16, 2020)

    • Adding Django 3.1 compatibility.
    • CachedStaticFilesStorage is removed from Django. Add a check of the current version to prevent error while importing. Thank to @vmsp
    • Context in django.template.base is removed from Django and not used anymore in django-pipeline.
    • Fixing widgets tests of django-pipeline due to Media.render_js change in Django. More information in Django ticket #31892
    Source code(tar.gz)
    Source code(zip)
  • 2.0.4(May 29, 2020)

  • 2.0.3(May 23, 2020)

    • Add subclass of ManifestStaticFilesStorage. Thank to @jhpinson
    • Change the documentation to use PipelineManifestStorage in configuration instead of PipelineCachedStorage now deprecated.
    • Change import MutableMapping from collections.abc. Thank to @colons
    • Fix Middleware to properly decode HTML. Thank to @gatsinski
    • Keep mimetypes as str. Thank to @benspaulding
    • Based on #642 add 'NonPackagingPipelineManifestStorage' and update the documentation: storages.rst. Thank to @kronion
    • Remove futures from pipeline setup.py requirements.
    Source code(tar.gz)
    Source code(zip)
  • 2.0.0(May 23, 2020)

    • Definitely drop the support of Python 2.
    • Drop support for Python 3.5 (not compatible with PEP 498).
    • Remove 'decorator.py' how was used for backward compatibility between python 2 and 3 for metaclass inheritance on PipelineFormMedia.
    • Replace 'format' by 'fstring' (PEP 498: Literal String Interpolation).
    • Remove of old imports form 'django.utils.six' and these fixes (1.7.0).
    • Remove tests of uncovered versions of Python and Django.
    • Replace tests for Pypy by Pypy3.
    • Explicitly specify when files are read / write in binary mode.
    • Set opening files for tests to deal with universal newlines.
    • Upgrade documentation version to 2.0 to follow the project version.
    Source code(tar.gz)
    Source code(zip)
  • 1.7.0(Dec 28, 2019)

    Release the last major version of django-pipeline working on Python 2.

    • Thank you for all the modifications made since version 1.6.14, which we cannot quote.
    • Apply an optimization for compilation. Thank to @blankser
    • Edit setup.py to follow the recommendation of the documentation. Thank to @shaneikennedy
    • Add tests for Django 3.0 and Python 3.8
    • Add alternatives imports for django.utils.six, who has been removed in Django 3.0
    Source code(tar.gz)
    Source code(zip)
  • 1.6.13(May 25, 2017)

    • Fix forward-slashed paths on Windows. Thanks to @etiago
    • Fix CSS URL detector to match quotes correctly. Thanks to @vskh
    • Add a compiler_options dict to compile, to allow passing options to custom compilers. Thanks to @sassanh
    • Verify support for Django 1.11. Thanks to @jwhitlock
    Source code(tar.gz)
    Source code(zip)
  • 1.6.12(Mar 13, 2017)

Asset management for Python web development.

Asset management application for Python web development - use it to merge and compress your JavaScript and CSS files. Documentation: https://webassets

Michael Elsdörfer 912 Dec 25, 2022
Compresses linked and inline javascript or CSS into a single cached file.

Django Compressor Django Compressor processes, combines and minifies linked and inline Javascript or CSS in a Django template into cacheable static fi

2.6k Jan 03, 2023
Peak.py - An awesome tool to keep you up about servers and websites status

An In-house API Powered GUI App To Check Server Stats, Generate Logs And More Features To Be Added Soon.

3 Feb 24, 2022
Manage your activities as well as possible.

ToDo-List Manage your activities as well as possible. Create workspace Create many boards for different topics Add unlimited tasks About me Full name:

Matin Ardestani 10 Sep 11, 2022
HTML minifier for Python frameworks (not only Django, despite the name).

django-htmlmin django-html is an HTML minifier for Python, with full support for HTML 5. It supports Django, Flask and many other Python web framework

Cobra Team 536 Dec 25, 2022
django-systemjs

Django SystemJS Django SystemJS brings the Javascript of tomorrow to Django, today. It leverages JSPM (https://jspm.io) to do the heavy lifting for yo

Sergei Maertens 42 Jan 11, 2022
Pipeline is an asset packaging library for Django.

Pipeline Pipeline is an asset packaging library for Django, providing both CSS and JavaScript concatenation and compression, built-in JavaScript templ

Jazzband 1.4k Dec 26, 2022
https://django-storages.readthedocs.io/

Installation Installing from PyPI is as easy as doing: pip install django-storages If you'd prefer to install from source (maybe there is a bugfix in

Josh Schneier 2.3k Jan 05, 2023
Photonix Photo Manager - a photo management application based on web technologies

A modern, web-based photo management server. Run it on your home server and it will let you find the right photo from your collection on any device. Smart filtering is made possible by object recogni

Photonix Photo Manager 1.5k Jan 01, 2023
Compresses linked and inline javascript or CSS into a single cached file.

Django Compressor Django Compressor processes, combines and minifies linked and inline Javascript or CSS in a Django template into cacheable static fi

2.6k Jan 03, 2023
A faster collectstatic command.

Collectfast A faster collectstatic command. Features Efficiently decide what files to upload using cached checksums Parallel file uploads Supported St

Anton Agestam 405 Dec 27, 2022
A django compressor tool that bundles css, js files to a single css, js file with webpack and updates your html files with respective css, js file path.

django-webpacker's documentation: Introduction: django-webpacker is a django compressor tool which bundles css, js files to a single css, js file with

MicroPyramid 72 Aug 18, 2022
Python bindings to webpack

Unmaintained This project is unmaintained as it's a complicated solution to a simple problem. You should try using either https://github.com/owais/dja

Mark Finger 62 Apr 15, 2022
Transparently use webpack with django

django-webpack-loader Read http://owaislone.org/blog/webpack-plus-reactjs-and-django/ for a detailed step by step guide on setting up webpack with dja

2.4k Jan 06, 2023