๐Ÿฐ Bunnybook ๐Ÿฐ A tiny social network (for bunnies), built with FastAPI and React+RxJs.

Related tags

CMSbunnybook
Overview

๐Ÿฐ Bunnybook ๐Ÿฐ

A tiny social network (for bunnies), built with FastAPI and React+RxJs.

Click here for live demo!

ย 

Included features:

  • ๐Ÿ’ฌ chat
  • ๐Ÿ”ด online/offline friends status
  • ๐Ÿ”ก "Is typing..." indicator
  • ๐Ÿ‘ฌ friend requests and suggestions
  • ๐Ÿ”” notifications
  • ๐Ÿ“ฎ posts
  • ๐Ÿ“ comments
  • ๐Ÿ“œ conversations history
  • ๐Ÿ‡ random profile picture generation
  • ๐Ÿ”’ authentication

Tech stack:

  • ๐Ÿ Python 3.8 + FastAPI
  • ๐Ÿ“” PostgreSQL 13 + async SQLAlchemy (Core) + asyncpg driver
  • ๐Ÿ”— Neo4j graph db for relationships between users and fast queries
  • ๐ŸŽฏ Redis for caching and Pub/Sub
  • โšก Socket.IO for chat and notifications
  • ๐Ÿ”‘ Jwt + refresh tokens rotation for authentication
  • ๐Ÿณ Docker + docker-compose to ease deployment and development

Feel free to contact me on LinkedIn for ideas, discussions and opportunities ๐Ÿ™‚ https://www.linkedin.com/in/pietro-bassi/

Scope of the project

What Bunnybook is: I created Bunnybook to have the opportunity to experiment with some technologies I wasn't familiar with (e.g. Neo4j). I love learning new ways to solve problems at scale and a small social network seemed a very good candidate to test a few interesting libraries and techniques.

What Bunnybook isn't: a production-ready social network with infinite scaling capabilities. Building a "production-ready social network" would require way more testing and refinement (e.g. some features scale while others don't, test coverage is not complete, chat is hidden in mobile-mode and the entire project is not particularly responsive etc.), but this is out of the scope of this small experiment.

Quick start

Requirements

Deploy Bunnybook on your local machine

Open a shell and clone this repository:
git clone https://github.com/pietrobassi/bunnybook.git

Navigate inside project root folder:
cd bunnybook

Start all services:
docker-compose up

After some time (first execution might be slow), open Chrome or Firefox and navigate to:
http://localhost

API documentation is hosted at: http://localhost:8000/docs and http://localhost:8000/redoc

Please note: after pulling new changes from this repo, remember to execute docker-compose up --build to recreate containers

Developing

Requirements

Development setup

  • Clone this repository, navigate inside root folder and execute docker-compose -f docker-compose-dev.yml up to start services (PostgreSQL, Neo4j, Redis, etc.)

If you have other services running on your machine, port collisions are possibile: check docker-compose-dev.yml to see which ports are shared with host machine

  • Navigate to "backend" folder and create a Python 3.8 virtual environment; activate it, execute pip install -r requirements.txt, run python init_db.py (to generate databases schemas and constraints) and then python main.py (backend default port: 8000)
  • Navigate to "frontend" folder and install npm packages with npm install; start React development server with npm start (frontend default port: 3000)

Navigate to: http://localhost:3000

Testing

To run integration tests, navigate to "backend" folder, activate virtualenv and execute:
pytest
This command brings up a new clean docker-compose environment (with services mapped on different ports so they don't collide with the ones declared inside docker-compose-dev.yml), executes integration tests and performs services teardown.

To execute faster tests during development phase without leveraging the ad-hoc docker-compose environment, run
DEV=true pytest
while all the services are up and running: this will pollute you development database a bit with some test data, but it is much faster

Architectural considerations

Authentication

Authentication is performed via JWT access tokens + JWT refresh tokens (which are rotated at every refresh and stored in the database in order to allow session banning): with appropriate secret-sharing mechanisms, this configuration prevents the need for backend services to call a stateful user session holder (e.g. Redis) on every API call.
Access tokens are saved in localStorage whilst refresh token are stored inside secure, HTTP only cookies.
Saving access tokens in localStorage is not ideal since it exposes them to XSS attacks, but this choice comes from early stages of development where I wanted to use the same access token to authenticate both API calls and websocket connections and I was leveraging a websocket library that wasn't able to read Cookies inside "on connect" event handler; now, since Socket.IO can access them, a nice and easy fix would be to store access tokens in secure Cookies as well.

Caching

There are 2 Redis instances - one for chat/notifications and one for caching - in order not to have a single point of failure: a crash of cache backend shouldn't affect messaging! This is also the reason why backend code ignores failed cache calls (it just logs them) and goes on querying PostgreSQL/Neo4j to preserve Bunnybook functionality.
Most of caching follows Cache-Aside strategy. To store paginated results, only basic Redis data structures have been used, although I've also made some tests with sorted sets (ranked by post/comment/message/... creation date) and more complex logic in order to handle frequently changing data collections.

Chat

Chat-related database tables have been designed to support group chats in the future, although currently only private chats (1-to-1) are implemented.
To implement friends online/offline status, every websocket periodically sends a "I'm online!" message that sets a key on Redis ("websockets:{profile_id}") with short expiration time; a job is in charge of updating connected users friends' statuses, performing an MGET on Redis, using the recipient's friends' ids as keys, which are conveniently stored in Socket.IO session.
The simpler approach of setting "online" status of a profile when its websocket completes authentication successfully and setting it as "offline" when it disconnects is not viable because 1) if the backend instance - to which the websocket is connected - crashes, the Redis key doesn't get cleared and the friend appears as "online" forever 2) it doesn't support multiple logins from different devices/browsers, because disconnecting from a single device would incorrectly show the user as "offline".

Databases

Neo4j have been introduced alongside PostgreSQL to handle profiles relationships in a convenient way, allowing fast queries like "bunnies you may know" or "mutual friends" that would have been complex and/or slow with a traditional RDBMS.
Neo4j Bolt driver doesn't natively expose an async interface, so all calls are executed in separate threads (taken from a thread pool) not to block the main event loop. HTTP APIs + httpx could have been used to avoid the need for run_in_executor calls, at the cost of less convenient responses parsing and slightly worse performance (tested: still a valid solution in my opinion).

For PostgreSQL access, SQLAlchemy have been used asynchronously in "Core" mode; async "ORM" mode is quite new and - to use it correctly - some precautions must be taken to prevent implicit IO. Generally speaking, for CRUD-like apps (like Bunnybook) that don't involve a lot of business logic and complex objects interactions, I prefer to avoid ORMs and stick with query-builder libraries to have more control over the generated queries.
PostgreSQL connection pooling via pgbouncer is currently not used since encode/databases under the hood creates an asyncpg-managed connection pool; in order to increase scalability, a NullPool-like pool object should be used, overriding current encode/databases PostgreSQL backend adapter implementation.
In addition, PostgreSQL "pg_trgm" module as well as "gin" indexes have been added to speed up text search queries.

Dependency Injection

I made large use of dependency injection techniques both in frontend (microsoft/tsyringe) and backend (injector), where I extended standard FastAPI DI framework to make it compatible with "injector" package using a small utility function. Although some people might consider the extensive use of DI in Python an overkill since you have module imports, I still prefer to use it to have better control over instances initialization, easier testing and clearer dependency graph.

Frontend

RxJs stores have been chosen to manage state on the frontend. Coming from an Angular 2+ background, I have experience with Reactive Programming and I love working with RxJs since I think it is a really great tool to solve frontend state management issues, so I came up with a custom implementation of Angular services layer, leveraging RxJs stores + microsoft/tsyringe Dependency Injection framework. I had a great experience with this combo and I think I will reuse it for future projects. I like (and worked with) Redux / ReduxToolkit as well, but for small-to-medium sized projects I prefer the RxJs solution a little bit more since it feels more lightweight to me, also because including RxJs in the project allows me to use all its features for other tasks (not only state management).

Model

Backend model layer is made of pydantic classes, following the Anemic Domain Model pattern. Pydantic is a great way to represent domain model, ensuring continuous data validation and enforcing type hints at runtime. Regarding Anemic Domain Model, some people consider it an "anti-pattern": I respectfully disagree. Both Rich Domain Model and Anemic Domain Model have their reason to exist, what really matters is knowing why you are choosing one over the other and being aware of the pros/cons of each conceptual model.

Development ideas and open issues

  • implement "Likes" feature
  • cache notifications and chat messages
  • reimplement chat database schema to use a more "modern" approach, in order to leverage PostgreSQL 9.6+ recursive queries
  • rate limit Socket.IO events
  • store JWT access tokens the same way as JWT refresh tokens (secure + HTTP only + same site Cookies)
  • take countermeasures against cache stampede (e.g. through locking)
  • improve "RESTfulness" of profiles API
  • better frontend error handling
  • cleanup frontend "chat" code, which is a little bit messy and overcomplicated
  • increase test coverage, adding more integration tests, as well as unit and functional tests
Backend routes and database for an abstract theoretical app that relates a database of courses, users, and assignments.

Backend routes and database for an abstract theoretical app that relates a database of courses, users, and assignments.

Sean Wiesner 0 Dec 27, 2021
Python scripts to interact with the CakeCMS API.

Python scripts to interact with the CakeCMS API. Installation of the python module Prerequisites The cakecms module has to be installed first. Install

Fabian Thomas 3 Jan 31, 2022
A plugin for Wagtail CMS, to have Icon Blocks (Fontawesome support)

WAGTAIL ICONIFY Library developed for Wagtail CMS, its purpose is to provide icon blocks from various libraries Special thanks to Alex Gleason, as wel

2 Jun 07, 2022
Ticket shop application for conferences, festivals, concerts, tech events, shows, exhibitions, workshops, barcamps, etc.

pretix Reinventing ticket presales, one ticket at a time. Project status & release cycle While there is always a lot to do and improve on, pretix by n

pretix 1.3k Jan 01, 2023
The lektor static file content management system

Lektor Lektor is a static website generator. It builds out an entire project from static files into many individual HTML pages and has a built-in admi

Lektor CMS 3.6k Dec 29, 2022
Crypt Wiki - VimWiki with added support for encryption/decryption

Crypt Wiki - VimWiki with added support for encryption/decryption This project is meant to solve an issue I have ran into recently. I wanted to have a

Adrian Costin 6 Dec 18, 2022
Django content management as it should be

Django content management as it should be. Documentation Read the full documentation or get a quick brief below. Install $ pip install djedi-cms Confi

5 Monkeys 75 Dec 13, 2022
A website (webapp) to get food recipes by recipes names & ingredients.

Ramy's tedbira A website (aka: webapp) to get food recipes by recipes names & ingredients that you have in your fridge, using Django-framework and Spo

Rami Berrekia 15 Dec 23, 2022
An open source CMS, in python and integrable in Django

Python CMS based on the Django Framework

Titouan Bรฉnard 0 Sep 10, 2021
A curated list of awesome packages, articles, and other cool resources from the Wagtail community.

Awesome Wagtail A curated list of awesome packages, articles, and other cool resources from the Wagtail community. Wagtail is a Python CMS powered by

Springload 1.7k Jan 03, 2023
Abilian Social Business Engine - an enterprise social networking / collaboration platform.

About Abilian SBE (Social Business Engine) is a platform for social business applications, and more specifically collaborative / enterprise 2.0 busine

Abilian open source projects 63 Dec 29, 2022
A Django content management system focused on flexibility and user experience

Wagtail is an open source content management system built on Django, with a strong community and commercial support. It's focused on user experience,

Wagtail 13.8k Jan 02, 2023
Ella is a CMS based on Python web framework Django with a main focus on high-traffic news websites and Internet magazines.

Ella CMS Ella is opensource CMS based on Django framework, designed for flexibility. It is composed from several modules: Ella core is the main module

295 Oct 16, 2022
plumi video sharing

December 2017 update We are moving tickets from the Plumi tracker (trac.plumi.org) here, for historical reasons. Plumi video sharing system Plumi is a

Plumi 111 Dec 15, 2022
Library Management system designed for managing and operating various aspects of a library

Proposal This proposal is written to clarify the project work of Information Systems, assigned to us as an individual task in order to tackle the case

Prawal 1 Oct 27, 2021
Open Source CRM based on Django

Django-CRM Django CRM is opensource CRM developed on django framework. It has all the basic features of CRM to start with. We welcome code contributio

MicroPyramid 1.4k Dec 31, 2022
VaporCMS - The greatest content management system that will never exist

The greatest content management system that will never exist Overview WordPress is a huge success but could it be done better? Maybe being mo

Andrew Dailey 4 Jan 06, 2022
Indico - A feature-rich event management system, made @ CERN, the place where the Web was born.

Indico Indico is: ? a general-purpose event management tool; ? fully web-based; ? feature-rich but also extensible through the use of plugins; โš–๏ธ O

Indico 1.4k Dec 31, 2022
An encylopedia that runs on Django as part of CS50w's coursework

Django Wiki As part of the CS50w course, this project aims to apply the use of Django together with HTML and CSS to replicate an encyclopedia. Require

Beckham 1 Oct 28, 2021
The easy-to-use and developer-friendly CMS

django CMS Open source enterprise content management system based on the Django framework and backed by the non-profit django CMS Association. Get inv

django CMS Association 9.1k Jan 08, 2023