Skip to content

iliadmitriev/taskcamp

Repository files navigation

Taskcamp

Unittests with coverage Build docker and push codecov

This software is used for educational and demonstrative purposes. It's a simple project management tool powered by Django Framework

Features:

  • Class-Based Views approach
  • Login, Sign Up, Recover (LoginView, FormView, UserCreationForm, PasswordResetForm, LogoutView, PasswordResetView, PasswordResetDoneView, PasswordResetConfirmView, PasswordResetCompleteView)
  • Custom Extended User model (AbstractUser, BaseUserManager, UserManager)
  • Permissions and Groups (LoginRequiredMixin, PermissionRequiredMixin)
  • Simple CRUD views (ListView, DetailView, CreateView, UpdateView, DeleteView)
  • File uploading
  • Statistics (TemplateView, View, Q, F, Count, FloatField, Cast, Case, When, Sum, Avg)
  • Forms (Form, ModelForm)
  • Admin page (ModelAdmin, TabularInline)
  • Template and layouts (include templates, blocks, custom 500, 404, 403 handlers)
  • Router urls (include, namespace, params)
  • Caching (memcached)
  • Async workers with celery
  • Localization and internationalization (with pluralization)
  • Timezone support (pytz)
  • Markdown syntax, Status highlight (Template tags)
  • DB router (master, slave)
  • Unittests with coverage
  • Uwsgi (with static and media serving)
  • Docker and docker-compose
  • kubernetes deploy

Components:

Install


Types of installation

  1. Bare metal
  2. Docker
  3. Docker-compose
  4. Kubernetes

Bare metal install

  1. install python3 and create virtual env
python3 -m venv venv
source venv/bin/activate
  1. install requirements
pip install -r requirements.txt
  1. put django secret key into file .env generate DJANGO_SECRET_KEY
echo DJANGO_SECRET_KEY=\'$(python3 -c 'from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())')\'  > .env

or just create test

echo DJANGO_SECRET_KEY=test_secret_key  > .env
  1. add connection data to .env file
cat > .env << __EOF__
RABBITMQ_HOST=localhost
RABBITMQ_PORT=5672
RABBITMQ_DEFAULT_USER=admin
RABBITMQ_DEFAULT_PASS=adminsecret
RABBITMQ_DEFAULT_VHOST=celery

POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_DB=taskcamp
POSTGRES_USER=taskcamp
POSTGRES_PASSWORD=secret

MEMCACHED_LOCATION=localhost:11211

EMAIL_HOST=localhost
EMAIL_PORT=1025
__EOF__
# if you need a debug be enabled
cat >>.env << __EOF__
DJANGO_DEBUG=True
__EOF__

#if you need to keep celery results 
>>.env << __EOF__
REDIS_RESULTS_BACKEND=redis://localhost:6379/0
__EOF__
  1. create and run postgresql, memcached, mailcatcher and rabbitmq instance (if needed)
docker run -d --name taskcamp-postgres --hostname taskcamp-postgres \
    -p 5432:5432 --env-file .env postgres:alpine
    
docker run -d -p 11211:11211 --name taskcamp-memcached memcached:alpine

docker run -d -p 15672:15672 -p 5672:5672 \
  --name taskcamp-rabbit --hostname taskcamp-rabbit \
  --env-file .env rabbitmq:management-alpine

docker run -d -p 1080:1080 -p 1025:1025 \
 --name taskcamp-mailcatcher iliadmitriev/mailcatcher
# if you enabled REDIS_RESULTS_BACKEND to store celery results
docker run -d --name taskcamp-redis --hostname taskcamp-redis \
 -p 6379:6379 redis:alpine
  1. export variables from .env file
export $(cat .env | xargs)
  1. create a db (run migrations)
python3 manage.py migrate --no-input
  1. compile messages
python3 manage.py compilemessages -i venv
  1. create superuser
python3 manage.py createsuperuser

Docker install

Prepare

  1. create .env file
>.env << __EOF__
RABBITMQ_HOST=taskcamp-rabbitmq
RABBITMQ_PORT=5672
RABBITMQ_DEFAULT_USER=admin
RABBITMQ_DEFAULT_PASS=adminsecret
RABBITMQ_DEFAULT_VHOST=celery

POSTGRES_HOST=taskcamp-postgres
POSTGRES_PORT=5432
POSTGRES_DB=taskcamp
POSTGRES_USER=taskcamp
POSTGRES_PASSWORD=secret

MEMCACHED_LOCATION=taskcamp-memcached:11211
REDIS_RESULTS_BACKEND=redis://taskcamp-redis:6379/0

EMAIL_HOST=taskcamp-mail
EMAIL_PORT=1025
__EOF__
  1. create network taskcamp-network
docker network create taskcamp-network
  1. create docker containers (this is just an example, you shouldn't run in production like this)
docker run -d --name taskcamp-postgres --hostname taskcamp-postgres \
    --env-file .env --network taskcamp-network postgres:14-alpine
    
docker run -d --name taskcamp-memcached --hostname taskcamp-memcached \
    --network taskcamp-network memcached:alpine

docker run -d \
  --name taskcamp-rabbitmq --hostname taskcamp-rabbitmq \
  --env-file .env --network taskcamp-network \
  rabbitmq:management-alpine

docker run -d --name taskcamp-mail --hostname taskcamp-mail \
  --network taskcamp-network -p 1080:1080 iliadmitriev/mailcatcher
 
docker run -d --name taskcamp-redis --hostname taskcamp-redis \
  --network taskcamp-network redis:alpine

Build and run

  1. build docker image taskcamp-python
docker build -t taskcamp-python -f Dockerfile ./
  1. run django web application
docker run -p 8000:8000 --env-file=.env -d --name=taskcamp-django \
  --hostname=taskcamp-django --network taskcamp-network taskcamp-python
  1. run celery
docker run --env-file=.env -d --name=taskcamp-celery --hostname=taskcamp-celery \
   --network taskcamp-network taskcamp-python python3 -m celery -A worker worker
  1. apply migrations
docker run --env-file=.env --rm -ti --network taskcamp-network taskcamp-python \
    python3 manage.py migrate
  1. create superuser
docker run --env-file=.env --rm -ti --network taskcamp-network taskcamp-python \
    python3 manage.py createsuperuser

Clean up

docker rm -f $(docker ps --filter name=^taskcamp -a -q)
docker network rm taskcamp-network
docker rmi taskcamp-python

Docker-compose install

  1. create .env environment variables file
>.env << __EOF__
RABBITMQ_HOST=taskcamp-rabbitmq
RABBITMQ_PORT=5672
RABBITMQ_DEFAULT_USER=admin
RABBITMQ_DEFAULT_PASS=adminsecret
RABBITMQ_DEFAULT_VHOST=celery

POSTGRES_HOST=taskcamp-postgres
POSTGRES_PORT=5432
POSTGRES_DB=taskcamp
POSTGRES_USER=taskcamp
POSTGRES_PASSWORD=secret

MEMCACHED_LOCATION=taskcamp-memcached:11211
REDIS_RESULTS_BACKEND=redis://taskcamp-redis:6379/0

EMAIL_HOST=taskcamp-mail
EMAIL_PORT=1025
__EOF__
  1. start docker-compose services
docker-compose up -d
  1. apply migrations
docker-compose exec django python3 manage.py migrate
  1. create superuser
docker-compose exec django python3 manage.py createsuperuser
  1. load test data if needed
cat data.json | docker-compose exec -T django python3 manage.py loaddata --format=json - 

Docker-compose clean up

docker-compose down --rmi all --volumes

Development


  1. set environment variables
DJANGO_DEBUG=True
  1. make migrations and migrate
python3 manage.py makemigrations
python3 manage.py migrate
  1. make messages
python3 manage.py makemessages -a -i venv
python3 manage.py compilemessages -i venv
  1. run
python3 manage.py runserver 0:8000
  1. run celery for emails and other async tasks
python3 -m celery -A worker worker
# or
celery -A worker worker
  1. run celery for emails and other async tasks
python3 -m celery -A worker worker

or

celery -A worker worker

with log level and queue

celery -A worker worker -l INFO -Q email

Testing

Run tests

  1. run all tests
python3 manage.py test
  1. run with keeping db in case of test fails
python3 manage.py test --keepdb
  1. run all tests with details
python3 manage.py test --verbosity=2

Run tests with coverage

  1. run with coverage
coverage run manage.py test --verbosity=2
  1. print report with missing lines and fail with error in case it's not fully covered
coverage report -m --fail-under=100