Frontent dev env (#247)

* Added frontend development files/environment

* More items-categories related removals

* Improvements in pages templates (inc. static pages)

* Improvements in video player

* Added empty home page message + cta

* Updates in media, playlist and management pages

* Improvements in material icons font loading

* Replaced media & playlists links in frontend dev-env

* frontend package version update

* chnaged frontend dev url port

* static files update

* Changed default position of theme switcher

* enabled frontend docker container

.dockerignore

@@ -0,0 +1,2 @@
+node_modules
+npm-debug.log

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+# These are supported funding model platforms
+
+github: [mgogoulos]

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,31 @@
+---
+name: Issue report
+about: Create a report to help us improve MediaCMS
+title: ''
+labels: 'issue: bug'
+assignees: mgogoulos
+
+---
+
+**Describe the issue**
+A clear and concise description of what the issue is.
+
+**To Reproduce**
+Steps to reproduce the issue:
+1. Go to ...
+2. Perform action ...
+3. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Environment (please complete the following information):**
+ - OS: [e.g. Ubuntu Linux]
+ - Installation method: [Docker install, or single server install]
+ - Browser, if applicable
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea
+title: ''
+labels: 'issue: enhancement'
+assignees: mgogoulos
+
+---
+
+**Describe the feature you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/pull_request_template.md

@@ -0,0 +1,10 @@
+## Description
+<!-- Describe the changes introduced by this PR for the reviewers to fully understand. -->
+
+
+## Steps
+<!-- Actions to be done pre and post deployment -->
+*Pre-deploy*
+
+*Post-deploy*
+

.github/workflows/lint_test.yml

@@ -0,0 +1,15 @@
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - uses: actions/setup-python@v2
+    - uses: pre-commit/action@v2.0.0
+      with:
+        token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/python.yml

@@ -0,0 +1,35 @@
+name: Python Tests
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v1
+
+    - name: Build the Stack
+      run:  docker-compose -f docker-compose-dev.yaml build
+
+    - name: Start containers
+      run: docker-compose -f docker-compose-dev.yaml up -d
+
+    - name: List containers
+      run: docker ps
+
+    - name: Sleep for 60 seconds
+      run: sleep 60s
+      shell: bash
+
+    - name: Run Django Tests
+      run:  docker-compose -f docker-compose-dev.yaml exec -T web pytest
+
+    - name: Tear down the Stack
+      run:  docker-compose -f docker-compose-dev.yaml down

.gitignore

@@ -0,0 +1,17 @@
+media_files/encoded/
+media_files/original/
+media_files/hls/
+media_files/chunks/
+media_files/uploads/
+postgres_data/
+celerybeat-schedule
+logs/
+pids/
+static/admin/
+static/ckeditor/
+static/debug_toolbar/
+static/mptt/
+static/rest_framework/
+static/drf-yasg
+cms/local_settings.py
+deploy/docker/local_settings.py

.mailmap

@@ -0,0 +1 @@
+Swift Ugandan <swiftugandan@gmail.com> <swiftugandan@gmail.com>

.pre-commit-config.yaml

@@ -0,0 +1,15 @@
+repos:
+  - repo: https://gitlab.com/pycqa/flake8
+    rev: 3.7.9
+    hooks:
+      - id: flake8
+  - repo: https://github.com/pycqa/isort
+    rev: 5.5.4
+    hooks:
+      - id: isort
+        args: ["--profile", "black"]
+  - repo: https://github.com/psf/black
+    rev: 20.8b1
+    hooks:
+      - id: black
+        language_version: python3  

AUTHORS.txt

@@ -2,3 +2,7 @@ Wordgames.gr - https://www.wordgames.gr
 Yiannis Stergiou - ys.stergiou@gmail.com
 Markos Gogoulos - mgogoulos@gmail.com
 
+Contributors
+
+Swift Ugandan - swiftugandan@gmail.com
+

CODE_OF_CONDUCT.md

@@ -0,0 +1,13 @@
+# Contributor Code of Conduct
+
+As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
+
+Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
+
+This Code of Conduct is adapted from the [Contributor Covenant](http:contributor-covenant.org), version 1.0.0, available at https://www.contributor-covenant.org/version/1/0/0/code-of-conduct.html

Dockerfile

@@ -0,0 +1,66 @@
+FROM python:3.8-buster AS compile-image
+
+SHELL ["/bin/bash", "-c"]
+
+# Set up virtualenv
+ENV VIRTUAL_ENV=/home/mediacms.io
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+ENV PIP_NO_CACHE_DIR=1
+
+RUN mkdir -p /home/mediacms.io/mediacms/{logs} && cd /home/mediacms.io && python3 -m venv $VIRTUAL_ENV
+
+# Install dependencies:
+COPY requirements.txt .
+RUN pip install -r requirements.txt
+
+COPY . /home/mediacms.io/mediacms
+WORKDIR /home/mediacms.io/mediacms
+
+RUN wget -q http://zebulon.bok.net/Bento4/binaries/Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip && \
+    unzip Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip -d ../bento4 && \
+    mv ../bento4/Bento4-SDK-1-6-0-637.x86_64-unknown-linux/* ../bento4/ && \
+    rm -rf ../bento4/Bento4-SDK-1-6-0-637.x86_64-unknown-linux && \
+    rm -rf ../bento4/docs && \
+    rm Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip
+
+############ RUNTIME IMAGE ############
+FROM python:3.8-slim-buster as runtime-image
+
+ENV PYTHONUNBUFFERED=1
+ENV PYTHONDONTWRITEBYTECODE=1
+ENV ADMIN_USER='admin'
+ENV ADMIN_PASSWORD='mediacms'
+ENV ADMIN_EMAIL='admin@localhost'
+
+# See: https://github.com/celery/celery/issues/6285#issuecomment-715316219
+ENV CELERY_APP='cms'
+
+# Use these to toggle which processes supervisord should run
+ENV ENABLE_UWSGI='yes'
+ENV ENABLE_NGINX='yes'
+ENV ENABLE_CELERY_BEAT='yes'
+ENV ENABLE_CELERY_SHORT='yes'
+ENV ENABLE_CELERY_LONG='yes'
+ENV ENABLE_MIGRATIONS='yes'
+
+# Set up virtualenv
+ENV VIRTUAL_ENV=/home/mediacms.io
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
+COPY --chown=www-data:www-data --from=compile-image /home/mediacms.io /home/mediacms.io
+
+RUN apt-get update -y && apt-get -y upgrade && apt-get install --no-install-recommends \
+    supervisor nginx ffmpeg imagemagick procps -y && \
+    rm -rf /var/lib/apt/lists/* && \
+    apt-get purge --auto-remove && \
+    apt-get clean
+
+WORKDIR /home/mediacms.io/mediacms
+
+EXPOSE 9000 80
+
+RUN chmod +x ./deploy/docker/entrypoint.sh
+
+ENTRYPOINT ["./deploy/docker/entrypoint.sh"]
+
+CMD ["./deploy/docker/start.sh"]

Dockerfile-dev

@@ -0,0 +1,16 @@
+FROM mediacms/mediacms:latest
+
+SHELL ["/bin/bash", "-c"]
+
+# Set up virtualenv
+ENV VIRTUAL_ENV=/home/mediacms.io
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+ENV PIP_NO_CACHE_DIR=1
+
+RUN cd /home/mediacms.io && python3 -m venv $VIRTUAL_ENV
+
+COPY requirements.txt . 
+COPY requirements-dev.txt .
+RUN pip install -r requirements-dev.txt
+
+WORKDIR /home/mediacms.io/mediacms

README.md

@@ -1,23 +1,28 @@
-![MediaCMS](static/images/logo_dark.png)
+# MediaCMS
+
+[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/python/g/mediacms-io/mediacms.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/mediacms-io/mediacms/context:python)
+[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/javascript/g/mediacms-io/mediacms.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/mediacms-io/mediacms/context:javascript)
+<br/>
+[![GitHub license](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://raw.githubusercontent.com/mediacms-io/mediacms/main/LICENSE.txt)
+[![Downloads](https://img.shields.io/badge/download-all%20releases-brightgreen.svg)](https://github.com/mediacms-io/mediacms/releases/)
+[![DockerHub](https://img.shields.io/docker/pulls/mediacms/mediacms)](https://hub.docker.com/repository/docker/mediacms/mediacms/)
+
+
 
 MediaCMS is a modern, fully featured open source video and media CMS. It is developed to meet the needs of modern web platforms for viewing and sharing media. It can be used to build a small to medium video and media portal within minutes. 
 
 It is built mostly using the modern stack Django + React and includes a REST API.
 
+A demo is available at https://demo.mediacms.io
+
+
 ## Screenshots
 
-![MediaCMS](docs/images/index.jpg)
-
-Vanilla MediaCMS index page
-
-![MediaCMS](docs/images/video.jpg)
-
-Video page with player different options
-
-![MediaCMS](docs/images/embed.jpg)
-
-Embed video page
-
+<p align="center">
+    <img src="https://raw.githubusercontent.com/mediacms-io/mediacms/main/docs/images/index.jpg" width="340">
+    <img src="https://raw.githubusercontent.com/mediacms-io/mediacms/main/docs/images/video.jpg" width="340">
+    <img src="https://raw.githubusercontent.com/mediacms-io/mediacms/main/docs/images/embed.jpg" width="340">
+</p>
 
 ## Features
 - **Complete control over your data**: host it yourself!
@@ -27,6 +32,7 @@ Embed video page
 - **Multiple media classification options**: categories, tags and custom
 - **Multiple media sharing options**: social media share, videos embed code generation
 - **Easy media searching**: enriched with live search functionality
+- **Playlists for audio and video content**: create playlists, add and reorder content
 - **Responsive design**: including light and dark themes
 - **Advanced users management**: allow self registration, invite only, closed.
 - **Configurable actions**: allow download, add comments, add likes, dislikes, report media
@@ -37,6 +43,7 @@ Embed video page
 - **Subtitles/CC**: support for multilingual subtitle files
 - **Scalable transcoding**: transcoding through priorities. Experimental support for remote workers
 - **Chunked file uploads**: for pausable/resumable upload of content
+- **REST API**: Documented through Swagger
 
 
 ## Example cases
@@ -76,85 +83,46 @@ For a small to medium installation, with a few hours of video uploaded daily, an
 In terms of disk space, think of what the needs will be. A general rule is to multiply by three the size of the expected uploaded videos (since the system keeps original versions, encoded versions plus HLS), so if you receive 1G of videos daily and maintain all of them, you should consider a 1T disk across a year (1G * 3 * 365).
 
 
-## Install
+## Installation / Maintanance
 
-The core dependencies are Python3, Django3, Celery, PostgreSQL, Redis, ffmpeg. Any system that can have these dependencies installed, can run MediaCMS. But we strongly suggest installing on Linux Ubuntu 18 or 20 versions.
+There are two ways to run MediaCMS, through Docker Compose and through installing it on a server via an automation script that installs and configures all needed services. Find the related pages:
 
-Installation on a Ubuntu 18 or 20 system with git utility installed should be completed in a few minutes with the following steps.
-Make sure you run it as user root, on a clear system, since the automatic script will install and configure the following services: Celery/PostgreSQL/Redis/Nginx and will override any existing settings. 
+* [Single Server](docs/Single_Server.md) page
+* [Docker Compose](docs/Docker_Compose.md) page
 
-Automated script - to run on Ubuntu 18 or Ubuntu 20 flavors only!
-
-```bash
-mkdir /home/mediacms.io && cd /home/mediacms.io/
-git clone https://github.com/mediacms-io/mediacms
-cd /home/mediacms.io/mediacms/ && bash ./install.sh
-```
-
-The script will ask if you have a URL where you want to deploy MediaCMS, otherwise it will use localhost. If you provide a URL, it will use Let's Encrypt service to install a valid ssl certificate. 
-
-
-## Update
-
-If you've used the above way to install MediaCMS, update with the following:
-
-```bash
-cd /home/mediacms.io/mediacms # enter mediacms directory
-source  /home/mediacms.io/bin/activate # use virtualenv
-git pull # update code
-python manage.py migrate # run Django migrations
-sudo systemctl restart mediacms celery_long celery_short # restart services
-```
-
-
-## Configure
+## Configuration
 
 Several options are available on cms/settings.py, most of the things that are allowed or should be disallowed are described there. It is advisable to override any of them by adding it to cms/local_settings.py. All configuration options will be documented gradually on the [Configuration](docs/Configuration.md) page.
 
-## Authors
-MediaCMS is developed by Yiannis Stergiou and Markos Gogoulos. We are Wordgames - https://wordgames.gr
-
 
 ## Technology
-This software uses the following list of awesome technologies:
-- Python
-- Django
-- Django Rest Framework
-- Celery
-- PostgreSQL
-- Redis
-- Nginx
-- uWSGI
-- React
-- Fine Uploader
-- video.js
-- FFMPEG
-- Bento4
+This software uses the following list of awesome technologies: Python, Django, Django Rest Framework, Celery, PostgreSQL, Redis, Nginx, uWSGI, React, Fine Uploader, video.js, FFMPEG, Bento4
 
 
 ## Who is using it
 
-- **EngageMedia** non-profit media, technology and culture organization - https://video.engagemedia.org
-
+- **Cinemata** non-profit media, technology and culture organization - https://cinemata.org
 - **Critical Commons** public media archive and fair use advocacy network - https://criticalcommons.org
-
 - **Heritales** International Heritage Film Festival - https://stage.heritales.org
 
 
-## Thanks To
+## How to contribute
 
-- **Anna Helme**, for such a great partnership all these years!
+If you like the project, here's a few things you can do
+- Hire us, for custom installations, training, support, maintenance work
+- Suggest us to others that are interested to hire us
+- Write a blog post/article about MediaCMS
+- Share on social media about the project
+- Open issues, participate on discussions, report bugs, suggest ideas
+- Star the project
+- Add functionality, work on a PR, fix an issue! 
 
-- **Steve Anderson**, for trusting us and helping the Wordgames team make this real.
-
-- **Andrew Lowenthal, King Catoy, Rezwan Islam** and the rest of the great team of [Engage Media](https://engagemedia.org). 
-
-- **Ioannis Korovesis, Ioannis Maistros, Diomidis Spinellis and Theodoros Karounos**, for their mentorship all these years, their contribution to science and the promotion of open source and free software technologies.
-
-- **Antonis Ikonomou**, for hosting us on the excellent [Innovathens](https://www.innovathens.gr) space.
-
-- **Werner Robitza**, for helping us with ffmpeg related stuff.
+## Developers info
 
+- API documentation through Swagger is available under /swagger URL of your installation -example https://demo.mediacms.io/swagger/
+- We're working on proper documentation for users, managers and developers, until then checkout what's available on the docs/ folder of this repository
+- Before you send a PR, make sure your code is properly formatted. For that, use `pre-commit install` to install a pre-commit hook and run `pre-commit run --all` and fix everything before you commit. This pre-commit will check for your code lint everytime you commit a code.
+- Checkout the [Code of conduct page](CODE_OF_CONDUCT.md) if you want to contribute to this repository
 
 ## Contact
-info@mediacms.io
+info@mediacms.io

actions/migrations/0002_mediaaction_media.py

@@ -1,7 +1,7 @@
 # Generated by Django 3.1.4 on 2020-12-01 07:12
 
-from django.db import migrations, models
 import django.db.models.deletion
+from django.db import migrations, models
 
 
 class Migration(migrations.Migration):

actions/migrations/0003_auto_20201201_0712.py

@@ -1,8 +1,8 @@
 # Generated by Django 3.1.4 on 2020-12-01 07:12
 
+import django.db.models.deletion
 from django.conf import settings
 from django.db import migrations, models
-import django.db.models.deletion
 
 
 class Migration(migrations.Migration):
@@ -35,8 +35,6 @@ class Migration(migrations.Migration):
         ),
         migrations.AddIndex(
             model_name="mediaaction",
-            index=models.Index(
-                fields=["session_key", "action"], name="actions_med_session_fac55a_idx"
-            ),
+            index=models.Index(fields=["session_key", "action"], name="actions_med_session_fac55a_idx"),
         ),
     ]

actions/models.py

@@ -1,6 +1,7 @@
 from django.db import models
-from users.models import User
+
 from files.models import Media
+from users.models import User
 
 USER_MEDIA_ACTIONS = (
     ("like", "Like"),
@@ -30,15 +31,11 @@ class MediaAction(models.Model):
         help_text="for not logged in users",
     )
 
-    action = models.CharField(
-        max_length=20, choices=USER_MEDIA_ACTIONS, default="watch"
-    )
+    action = models.CharField(max_length=20, choices=USER_MEDIA_ACTIONS, default="watch")
     # keeps extra info, eg on report action, why it is reported
     extra_info = models.TextField(blank=True, null=True)
 
-    media = models.ForeignKey(
-        Media, on_delete=models.CASCADE, related_name="mediaactions"
-    )
+    media = models.ForeignKey(Media, on_delete=models.CASCADE, related_name="mediaactions")
     action_date = models.DateTimeField(auto_now_add=True)
     remote_ip = models.CharField(max_length=40, blank=True, null=True)
 

cms/__init__.py

@@ -1,4 +1,5 @@
 from __future__ import absolute_import
+
 from .celery import app as celery_app
 
 __all__ = ["celery_app"]

cms/celery.py

@@ -1,5 +1,7 @@
 from __future__ import absolute_import
+
 import os
+
 from celery import Celery
 
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cms.settings")

cms/custom_pagination.py

@@ -1,8 +1,9 @@
-from rest_framework.pagination import PageNumberPagination
-from rest_framework.response import Response
 from collections import OrderedDict  # requires Python 2.7 or later
+
 from django.core.paginator import Paginator
 from django.utils.functional import cached_property
+from rest_framework.pagination import PageNumberPagination
+from rest_framework.response import Response
 
 
 class FasterDjangoPaginator(Paginator):

cms/permissions.py

@@ -1,5 +1,6 @@
 from django.conf import settings
 from rest_framework import permissions
+
 from files.methods import is_mediacms_editor, is_mediacms_manager
 
 
@@ -24,7 +25,10 @@ class IsUserOrManager(permissions.BasePermission):
         if is_mediacms_manager(request.user):
             return True
 
-        return obj.user == request.user
+        if hasattr(obj, 'user'):
+            return obj.user == request.user
+        else:
+            return obj == request.user
 
 
 class IsUserOrEditor(permissions.BasePermission):

cms/settings.py

@@ -1,4 +1,5 @@
 import os
+
 from celery.schedules import crontab
 
 DEBUG = False
@@ -16,14 +17,15 @@ CAN_ADD_MEDIA = "all"
 # valid choices here are 'public', 'private', 'unlisted
 PORTAL_WORKFLOW = "public"
 
-DEFAULT_THEME = "black"  # this is not taken under consideration currently
+# valid values: 'light', 'dark'.
+DEFAULT_THEME = "light"
 
 
 # These are passed on every request
 # if set to False will not fetch external content
 # this is only for the static files, as fonts/css/js files loaded from CDNs
 # not for user uploaded media!
-LOAD_FROM_CDN = True
+LOAD_FROM_CDN = False
 LOGIN_ALLOWED = True  # whether the login button appears
 REGISTER_ALLOWED = True  # whether the register button appears
 UPLOAD_MEDIA_ALLOWED = True  # whether the upload media button appears
@@ -42,7 +44,11 @@ ALLOW_RATINGS_CONFIRMED_EMAIL_ONLY = True
 
 # ip of the server should be part of this
 ALLOWED_HOSTS = ["*", "mediacms.io", "127.0.0.1", "localhost"]
+
 FRONTEND_HOST = "http://localhost"
+# this variable - along with SSL_FRONTEND_HOST is used on several places
+# as email where a URL need appear etc
+
 # FRONTEND_HOST needs an http prefix - at the end of the file
 # there's a conversion to https with the SSL_FRONTEND_HOST env
 INTERNAL_IPS = "127.0.0.1"
@@ -208,9 +214,7 @@ POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY = ""
 CANNOT_ADD_MEDIA_MESSAGE = ""
 
 # mp4hls command, part of Bendo4
-MP4HLS_COMMAND = (
-    "/home/mediacms.io/mediacms/Bento4-SDK-1-6-0-632.x86_64-unknown-linux/bin/mp4hls"
-)
+MP4HLS_COMMAND = "/home/mediacms.io/mediacms/Bento4-SDK-1-6-0-637.x86_64-unknown-linux/bin/mp4hls"
 
 # highly experimental, related with remote workers
 ADMIN_TOKEN = "c2b8e1838b6128asd333ddc5e24"
@@ -288,6 +292,7 @@ INSTALLED_APPS = [
     "uploader.apps.UploaderConfig",
     "djcelery_email",
     "ckeditor",
+    "drf_yasg",
 ]
 
 MIDDLEWARE = [
@@ -344,6 +349,16 @@ FILE_UPLOAD_HANDLERS = [
 
 LOGS_DIR = os.path.join(BASE_DIR, "logs")
 
+error_filename = os.path.join(LOGS_DIR, "debug.log")
+if not os.path.exists(LOGS_DIR):
+    try:
+        os.mkdir(LOGS_DIR)
+    except PermissionError:
+        pass
+
+if not os.path.isfile(error_filename):
+    open(error_filename, 'a').close()
+
 LOGGING = {
     "version": 1,
     "disable_existing_loggers": False,
@@ -351,7 +366,7 @@ LOGGING = {
         "file": {
             "level": "ERROR",
             "class": "logging.FileHandler",
-            "filename": os.path.join(LOGS_DIR, "debug.log"),
+            "filename": error_filename,
         },
     },
     "loggers": {
@@ -419,9 +434,16 @@ CELERY_BEAT_SCHEDULE = {
 # TODO: beat, delete chunks from media root
 # chunks_dir after xx days...(also uploads_dir)
 
+
+LOCAL_INSTALL = False
+
+# this is an option to make the whole portal available to logged in users only
+# it is placed here so it can be overrided on local_settings.py
+GLOBAL_LOGIN_REQUIRED = False
+
 try:
     # keep a local_settings.py file for local overrides
-    from .local_settings import *
+    from .local_settings import *  # noqa
 
     # ALLOWED_HOSTS needs a url/ip
     ALLOWED_HOSTS.append(FRONTEND_HOST.replace("http://", "").replace("https://", ""))
@@ -434,4 +456,16 @@ if "http" not in FRONTEND_HOST:
     # FRONTEND_HOST needs a http:// preffix
     FRONTEND_HOST = f"http://{FRONTEND_HOST}"
 
-SSL_FRONTEND_HOST = FRONTEND_HOST.replace("http", "https")
+if LOCAL_INSTALL:
+    SSL_FRONTEND_HOST = FRONTEND_HOST.replace("http", "https")
+else:
+    SSL_FRONTEND_HOST = FRONTEND_HOST
+
+if GLOBAL_LOGIN_REQUIRED:
+    # this should go after the AuthenticationMiddleware middleware
+    MIDDLEWARE.insert(5, "login_required.middleware.LoginRequiredMiddleware")
+    LOGIN_REQUIRED_IGNORE_PATHS = [
+        r'/accounts/login/$',
+        r'/accounts/logout/$',
+        r'/accounts/signup/$',
+    ]

cms/urls.py

@@ -1,13 +1,31 @@
-from django.contrib import admin
-from django.urls import path
-from django.conf.urls import url, include
 import debug_toolbar
+from django.conf.urls import include, url
+from django.contrib import admin
+from django.urls import path, re_path
+from django.views.generic.base import TemplateView
+from drf_yasg import openapi
+from drf_yasg.views import get_schema_view
+from rest_framework.permissions import AllowAny
+
+schema_view = get_schema_view(
+    openapi.Info(title="MediaCMS API", default_version='v1', contact=openapi.Contact(url="https://mediacms.io"), x_logo={"url": "../../static/images/logo_dark.svg"}),
+    public=True,
+    permission_classes=(AllowAny,),
+)
+
 
 urlpatterns = [
     url(r"^__debug__/", include(debug_toolbar.urls)),
+    path(
+        "robots.txt",
+        TemplateView.as_view(template_name="robots.txt", content_type="text/plain"),
+    ),
     url(r"^", include("files.urls")),
     url(r"^", include("users.urls")),
     url(r"^accounts/", include("allauth.urls")),
     url(r"^api-auth/", include("rest_framework.urls")),
     path("admin/", admin.site.urls),
+    re_path(r'^swagger(?P<format>\.json|\.yaml)$', schema_view.without_ui(cache_timeout=0), name='schema-json'),
+    re_path(r'^swagger/$', schema_view.with_ui('swagger', cache_timeout=0), name='schema-swagger-ui'),
+    path('docs/api/', schema_view.with_ui('redoc', cache_timeout=0), name='schema-redoc'),
 ]

conftest.py

@@ -0,0 +1,5 @@
+from pytest_factoryboy import register
+
+from tests.users.factories import UserFactory
+
+register(UserFactory)

deploy/docker/README.md

@@ -0,0 +1,3 @@
+# MediaCMS on Docker
+
+See: [Details](../../docs/Docker_deployment.md)

deploy/docker/entrypoint.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+set -e
+
+# forward request and error logs to docker log collector
+ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log && \
+ln -sf /dev/stdout /var/log/nginx/mediacms.io.access.log && ln -sf /dev/stderr /var/log/nginx/mediacms.io.error.log
+
+cp /home/mediacms.io/mediacms/deploy/docker/local_settings.py /home/mediacms.io/mediacms/cms/local_settings.py
+
+mkdir -p /home/mediacms.io/mediacms/{logs,media_files/hls}
+touch /home/mediacms.io/mediacms/logs/debug.log
+
+mkdir -p /var/run/mediacms
+chown www-data:www-data /var/run/mediacms
+
+TARGET_GID=$(stat -c "%g" /home/mediacms.io/mediacms/)
+
+EXISTS=$(cat /etc/group | grep $TARGET_GID | wc -l)
+
+# Create new group using target GID and add www-data user
+if [ $EXISTS == "0" ]; then
+    groupadd -g $TARGET_GID tempgroup
+    usermod -a -G tempgroup www-data
+else
+    # GID exists, find group name and add
+    GROUP=$(getent group $TARGET_GID | cut -d: -f1)
+    usermod -a -G $GROUP www-data
+fi
+
+# We should do this only for folders that have a different owner, since it is an expensive operation
+find /home/mediacms.io/ ! \( -user www-data -group $TARGET_GID \) -exec chown www-data:$TARGET_GID {} +
+
+chmod +x /home/mediacms.io/mediacms/deploy/docker/start.sh /home/mediacms.io/mediacms/deploy/docker/prestart.sh
+
+exec "$@"

deploy/docker/local_settings.py

@@ -0,0 +1,34 @@
+FRONTEND_HOST = 'http://localhost'
+PORTAL_NAME = 'MediaCMS'
+SECRET_KEY = 'ma!s3^b-cw!f#7s6s0m3*jx77a@riw(7701**(r=ww%w!2+yk2'
+POSTGRES_HOST = 'db'
+REDIS_LOCATION = "redis://redis:6379/1"
+
+DATABASES = {
+    "default": {
+        "ENGINE": "django.db.backends.postgresql",
+        "NAME": "mediacms",
+        "HOST": POSTGRES_HOST,
+        "PORT": "5432",
+        "USER": "mediacms",
+        "PASSWORD": "mediacms",
+    }
+}
+
+CACHES = {
+    "default": {
+        "BACKEND": "django_redis.cache.RedisCache",
+        "LOCATION": REDIS_LOCATION,
+        "OPTIONS": {
+            "CLIENT_CLASS": "django_redis.client.DefaultClient",
+        },
+    }
+}
+
+# CELERY STUFF
+BROKER_URL = REDIS_LOCATION
+CELERY_RESULT_BACKEND = BROKER_URL
+
+MP4HLS_COMMAND = "/home/mediacms.io/bento4/bin/mp4hls"
+
+DEBUG = False

deploy/docker/nginx_http_only.conf

@@ -0,0 +1,30 @@
+server {
+    listen 80 ;
+
+    gzip on;
+    access_log /var/log/nginx/mediacms.io.access.log;
+
+    error_log  /var/log/nginx/mediacms.io.error.log  warn;
+
+    location /static {
+        alias /home/mediacms.io/mediacms/static ;
+    }
+
+    location /media/original {
+        alias /home/mediacms.io/mediacms/media_files/original;
+    }
+
+    location /media {
+        alias /home/mediacms.io/mediacms/media_files ;
+    }
+
+    location / {
+        add_header 'Access-Control-Allow-Origin' '*';
+        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
+
+        include /etc/nginx/sites-enabled/uwsgi_params;
+        uwsgi_pass 127.0.0.1:9000;
+    }
+}

deploy/docker/prestart.sh

@@ -0,0 +1,61 @@
+#!/bin/bash
+
+RANDOM_ADMIN_PASS=`python -c "import secrets;chars = 'abcdefghijklmnopqrstuvwxyz0123456789';print(''.join(secrets.choice(chars) for i in range(10)))"`
+ADMIN_PASSWORD=${ADMIN_PASSWORD:-$RANDOM_ADMIN_PASS}
+
+if [ X"$ENABLE_MIGRATIONS" = X"yes" ]; then
+    python manage.py migrate
+    python manage.py loaddata fixtures/encoding_profiles.json
+    python manage.py loaddata fixtures/categories.json
+    python manage.py collectstatic --noinput
+
+    echo "Admin Password: $ADMIN_PASSWORD"
+
+    # post_save, needs redis to succeed (ie. migrate depends on redis)
+    DJANGO_SUPERUSER_PASSWORD=$ADMIN_PASSWORD python manage.py createsuperuser \
+        --no-input \
+        --username=$ADMIN_USER \
+        --email=$ADMIN_EMAIL \
+        --database=default || true
+
+    # echo "Updating hostname ..."
+    # TODO: Get the FRONTEND_HOST from cms/local_settings.py
+    # echo "from django.contrib.sites.models import Site; Site.objects.update(name='$FRONTEND_HOST', domain='$FRONTEND_HOST')" | python manage.py shell
+fi
+
+# Setting up internal nginx server
+# HTTPS setup is delegated to a reverse proxy running infront of the application
+
+cp deploy/docker/nginx_http_only.conf /etc/nginx/sites-available/default
+cp deploy/docker/nginx_http_only.conf /etc/nginx/sites-enabled/default
+cp deploy/docker/uwsgi_params /etc/nginx/sites-enabled/uwsgi_params
+cp deploy/docker/nginx.conf /etc/nginx/
+
+#### Supervisord Configurations #####
+
+cp deploy/docker/supervisord/supervisord-debian.conf /etc/supervisor/conf.d/supervisord-debian.conf
+
+if [ X"$ENABLE_UWSGI" = X"yes" ] ; then
+    echo "Enabling uwsgi app server"
+    cp deploy/docker/supervisord/supervisord-uwsgi.conf /etc/supervisor/conf.d/supervisord-uwsgi.conf
+fi
+
+if [ X"$ENABLE_NGINX" = X"yes" ] ; then
+    echo "Enabling nginx as uwsgi app proxy and media server"
+    cp deploy/docker/supervisord/supervisord-nginx.conf /etc/supervisor/conf.d/supervisord-nginx.conf
+fi
+
+if [ X"$ENABLE_CELERY_BEAT" = X"yes" ] ; then
+    echo "Enabling celery-beat scheduling server"
+    cp deploy/docker/supervisord/supervisord-celery_beat.conf /etc/supervisor/conf.d/supervisord-celery_beat.conf
+fi
+
+if [ X"$ENABLE_CELERY_SHORT" = X"yes" ] ; then
+    echo "Enabling celery-short task worker"
+    cp deploy/docker/supervisord/supervisord-celery_short.conf /etc/supervisor/conf.d/supervisord-celery_short.conf
+fi
+
+if [ X"$ENABLE_CELERY_LONG" = X"yes" ] ; then
+    echo "Enabling celery-long task worker"
+    cp deploy/docker/supervisord/supervisord-celery_long.conf /etc/supervisor/conf.d/supervisord-celery_long.conf
+fi

deploy/docker/reverse_proxy/certs/localhost.crt

@@ -0,0 +1,17 @@
+-----BEGIN CERTIFICATE-----
+MIICwzCCAaugAwIBAgIJAOyvdwguJQd+MA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
+BAMTCWxvY2FsaG9zdDAeFw0yMTAxMjQxMjUwMzFaFw0zMTAxMjIxMjUwMzFaMBQx
+EjAQBgNVBAMTCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
+ggEBAONswEwBzkgoO+lkewiKUnwvYqC54qleCUg9hidqjoyzd5XWKh1mIF7aaSCG
+rJGSxCce8CbqAqGkpvsgXzwwbY72l7FwmAXFHO5ObQfpmFhjt2fsKRM9MTCo/UyU
+liuhgP+Q+BNzUontTUC40NVHs8R7IHG4z8unB7qB/7zGK2tfilLB8JDqPTkc22vN
+C4P1YxiGyY5bm37wQrroC9zPJ8bqanrF9Y90QJHubibnPWqnZvK2HkDWjp5LYkn8
+IuzBycs1cLd8eMjU9aT72kweykvnGDDc3YbXFzT2zBTGSFEBROsVdPrNF9PaeE3j
+pu4UZ8Ge3Fp3VYd+04DnWtbQq0MCAwEAAaMYMBYwFAYDVR0RBA0wC4IJbG9jYWxo
+b3N0MA0GCSqGSIb3DQEBBQUAA4IBAQAdm2aGn4evosbdWgBHgzr6oYWBIiPpf1SA
+GXizuf5OaMActFP0rZ0mogndLH5d51J2qqSfOtaWSA5qwlPvDSTn1nvJeHoVLfZf
+kQHaB7/DaOPGsZCQBELPhYHwl7+Ej3HYE+siiaRfjC2NVgf8P/pAsTlKbe2e+34l
+GwWSFol24w5xAmUezCF41JiZbqHoZhSh7s/PuJnK2RvhpjkrIot8GvxnbvOcKDIv
+JzEKo3qPq8pc5RBkpP7Kp2+EgAYn1xAn0CekxZracW/MY+tg2mCeFucZW2V1iwVs
+LpAw6GJnjYz5mbrQskPbrJ9t78JGUKQ0kL/VUTfryUHMHYCiJlvd
+-----END CERTIFICATE-----

deploy/docker/reverse_proxy/certs/localhost.key

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpAIBAAKCAQEA42zATAHOSCg76WR7CIpSfC9ioLniqV4JSD2GJ2qOjLN3ldYq
+HWYgXtppIIaskZLEJx7wJuoCoaSm+yBfPDBtjvaXsXCYBcUc7k5tB+mYWGO3Z+wp
+Ez0xMKj9TJSWK6GA/5D4E3NSie1NQLjQ1UezxHsgcbjPy6cHuoH/vMYra1+KUsHw
+kOo9ORzba80Lg/VjGIbJjlubfvBCuugL3M8nxupqesX1j3RAke5uJuc9aqdm8rYe
+QNaOnktiSfwi7MHJyzVwt3x4yNT1pPvaTB7KS+cYMNzdhtcXNPbMFMZIUQFE6xV0
++s0X09p4TeOm7hRnwZ7cWndVh37TgOda1tCrQwIDAQABAoIBAQCmKKyOW7tlCNBN
+AzbI1JbTWKOMnoM2DxhlCV5cqgOgVPcIKEL428bGxniMZRjr+vkJRBddtxdZFj1R
+uSMbjJ5fF1dZMtQ/UvaCPhZ283p1CdXUPbz863ZnAPCf5Oea1RK0piw5ucYSM6h/
+owgg65Qx92uK6uYW+uAwqg440+ihNvnaZoVTx5CjZbL9KISkrlNJnuYiB5vzOD0i
+UVklO5Qz8VCuOcOVGZCA2SxHm4HAbg/aiQnpaUa9de4TsZ4ygF66pZh77T0wNOos
+sS1riKtHQpX+osJyoTI/rIKFAhycsZ+AA7Qpu6GW4xQlNS6K8vRiIbktwkC+IT0O
+RSn8Dg7BAoGBAPe5R8SpgXx9jKdA1eFa/Vjx5bmB96r2MviIOIWF8rs2K33xe+rj
+v+BZ2ZjdpVjcm2nRMf9r/eDq2ScNFWmKoZsUmdyT84Qq9yLcTSUdno+zCy+L0LNH
+DqJq5jIxJaV7amHeR/w10BVuiDmzhSsTmhfnXTUGRO/h2PjRyC3yEYdxAoGBAOsF
+2+gTsdOGlq6AVzW5MLZkreq8WCU2wWpZRiCPh6HJa8htuynYxO5AWUiNUbYKddj2
+0za9DFiXgH+Oo8wrkTYLEdN0T5/o+ScL5t3VG3m9R6pnuudLC2vmGQP0hNuZUpnF
+7FzdJ85h6taR2bM1zFzOfl81K0BhTHGxTU2r70vzAoGAVXuLJ3LyqtnMKn72DzDN
+0d6PTkdqBoW0qwyerHy/eRjFQ02MXE7BDJMUwmphv1tJCefVX/WNAwsnahFavTPI
+dnJSccpgMtB8vXvV5yPkbmPzTTHrD6JKi4Nl8hYBjqwa1rDUmFSdfHfK7FZlcqrt
+9qexAzYpnbmKnLoPYMNyhxECgYEAm5OCUeuPoL2MS7GLiXWwyFx3QFczZlcLzBGS
+uYUpvLBwF/qDlhz3p9uS/tMFzyK3hktF4Ate+9o2ZroOtd31PzgusbJh7zIylGVt
+i1VB3eGtaiFGeUuVIPTthE++Dvw80KxTXdnMOvNYmHduDBLF2H2c6/tvSSvfhbdf
+u9XgD38CgYAiLcVySxMKNpsXatuC31wjT+rnaH22SD/7pXe2q6MRW/s+bGOspu0v
+NeJSLoM98v8F99q0W0lgqesYJVI20Frru0DfXIp60ryaDolzve3Iwk8SOJUlcnUG
+cCtmPUkjyr18QAlrcCB4PozJGjpPWyabaY8gGwo8wAEpJWHrIJlHew==
+-----END RSA PRIVATE KEY-----

deploy/docker/reverse_proxy/client_max_body_size.conf

@@ -0,0 +1 @@
+client_max_body_size 5800M;

deploy/docker/start.sh

@@ -0,0 +1,17 @@
+#! /usr/bin/env sh
+set -e
+
+# If there's a prestart.sh script in the /app directory, run it before starting
+PRE_START_PATH=deploy/docker/prestart.sh
+echo "Checking for script in $PRE_START_PATH"
+if [ -f $PRE_START_PATH ] ; then
+    echo "Running script $PRE_START_PATH"
+    . $PRE_START_PATH
+else
+    echo "There is no script $PRE_START_PATH"
+fi
+
+# Start Supervisor, with Nginx and uWSGI
+echo "Starting server using supervisord..."
+
+exec /usr/bin/supervisord

deploy/docker/supervisord/supervisord-celery_beat.conf

@@ -0,0 +1,12 @@
+[program:celery_beat]
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0
+startsecs=0
+numprocs=1
+user=www-data
+directory=/home/mediacms.io/mediacms
+priority=300
+startinorder=true
+command=/home/mediacms.io/bin/celery beat --pidfile=/var/run/mediacms/beat%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_beat.log

deploy/docker/supervisord/supervisord-celery_long.conf

@@ -0,0 +1,13 @@
+[program:celery_long]
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0
+startsecs=10
+numprocs=1
+user=www-data
+directory=/home/mediacms.io/mediacms
+priority=500
+startinorder=true
+startsecs=0
+command=/home/mediacms.io/bin/celery multi start long1 --pidfile=/var/run/mediacms/%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_long.log -Ofair --prefetch-multiplier=1 -Q long_tasks

deploy/docker/supervisord/supervisord-celery_short.conf

@@ -0,0 +1,12 @@
+[program:celery_short]
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0
+startsecs=0
+numprocs=1
+user=www-data
+directory=/home/mediacms.io/mediacms
+priority=400
+startinorder=true
+command=/home/mediacms.io/bin/celery multi start short1 short2 --pidfile=/var/run/mediacms/%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_short.log --soft-time-limit=300 -c10 -Q short_tasks

deploy/docker/supervisord/supervisord-debian.conf

@@ -0,0 +1,2 @@
+[supervisord]
+nodaemon=true

deploy/docker/supervisord/supervisord-nginx.conf

@@ -0,0 +1,11 @@
+[program:nginx]
+command=/usr/sbin/nginx -g 'daemon off;'
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0
+priority=200
+startinorder=true
+startsecs=0
+# Graceful stop, see http://nginx.org/en/docs/control.html
+stopsignal=QUIT

deploy/docker/supervisord/supervisord-uwsgi.conf

@@ -0,0 +1,9 @@
+[program:uwsgi]
+command=/home/mediacms.io/bin/uwsgi --ini /home/mediacms.io/mediacms/deploy/docker/uwsgi.ini
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0
+priority=100
+startinorder=true
+startsecs=0

deploy/docker/uwsgi.ini

@@ -0,0 +1,23 @@
+[uwsgi]
+
+chdir = /home/mediacms.io/mediacms/
+virtualenv = /home/mediacms.io
+module = cms.wsgi
+
+uid=www-data
+gid=www-data
+
+processes = 2
+threads = 2
+
+master = true
+
+socket = 127.0.0.1:9000
+
+workers = 2
+
+vacuum = true
+
+hook-master-start = unix_signal:15 gracefully_kill_them_all
+need-app = true
+die-on-term = true

deploy/local_install/dhparams.pem

@@ -0,0 +1,13 @@
+-----BEGIN DH PARAMETERS-----
+MIICCAKCAgEAo3MMiEY/fNbu+usIM0cDi6x8G3JBApv0Lswta4kiyedWT1WN51iQ
+9zhOFpmcu6517f/fR9MUdyhVKHxxSqWQTcmTEFtz4P3VLTS/W1N5VbKE2VEMLpIi
+wr350aGvV1Er0ujcp5n4O4h0I1tn4/fNyDe7+pHCdwM+hxe8hJ3T0/tKtad4fnIs
+WHDjl4f7m7KuFfheiK7Efb8MsT64HDDAYXn+INjtDZrbE5XPw20BqyWkrf07FcPx
+8o9GW50Ox7/FYq7jVMI/skEu0BRc8u6uUD9+UOuWUQpdeHeFcvLOgW53Z03XwWuX
+RXosUKzBPuGtUDAaKD/HsGW6xmGr2W9yRmu27jKpfYLUb/eWbbnRJwCw04LdzPqv
+jmtq02Gioo3lf5H5wYV9IYF6M8+q/slpbttsAcKERimD1273FBRt5VhSugkXWKjr
+XDhoXu6vZgj8Opei38qPa8pI1RUFoXHFlCe6WpZQmU8efL8gAMrJr9jUIY8eea1n
+u20t5B9ueb9JMjrNafcq6QkKhZLi6fRDDTUyeDvc0dN9R/3Yts97SXfdi1/lX7HS
+Ht4zXd5hEkvjo8GcnjsfZpAC39QfHWkDaeUGEqsl3jXjVMfkvoVY51OuokPWZzrJ
+M5+wyXNpfGbH67dPk7iHgN7VJvgX0SYscDPTtms50Vk7RwEzLeGuSHMCAQI=
+-----END DH PARAMETERS-----

deploy/local_install/mediacms.io

@@ -7,15 +7,15 @@ server {
 
     error_log  /var/log/nginx/mediacms.io.error.log  warn;
 
-    # redirect to https if logged in
-    if ($http_cookie ~* "sessionid") {
-        rewrite  ^/(.*)$  https://localhost/$1  permanent;
-    }
+#    # redirect to https if logged in
+#    if ($http_cookie ~* "sessionid") {
+#        rewrite  ^/(.*)$  https://localhost/$1  permanent;
+#    }
 
-    # redirect basic forms to https
-    location ~ (login|login_form|register|mail_password_form)$ {
-        rewrite  ^/(.*)$  https://localhost/$1  permanent;
-    }
+#    # redirect basic forms to https
+#    location ~ (login|login_form|register|mail_password_form)$ {
+#        rewrite  ^/(.*)$  https://localhost/$1  permanent;
+#    }
 
     location /static {
         alias /home/mediacms.io/mediacms/static ;
@@ -46,6 +46,12 @@ server {
 
     ssl_certificate_key  /etc/letsencrypt/live/localhost/privkey.pem;
     ssl_certificate  /etc/letsencrypt/live/localhost/fullchain.pem;
+    ssl_dhparam /etc/nginx/dhparams/dhparams.pem;
+
+    ssl_protocols TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
+    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;    
+    ssl_ecdh_curve secp521r1:secp384r1;
+    ssl_prefer_server_ciphers on;
 
     gzip on;
     access_log /var/log/nginx/mediacms.io.access.log;
@@ -59,7 +65,7 @@ server {
     location /media/original {
         alias /home/mediacms.io/mediacms/media_files/original;
         #auth_basic "auth protected area";
-        #auth_basic_user_file /home/mediacms.io/mediacms/deploy/.htpasswd;
+        #auth_basic_user_file /home/mediacms.io/mediacms/deploy/local_install/.htpasswd;
     }
 
     location /media {

deploy/local_install/mediacms.service

@@ -2,7 +2,7 @@
 Description=MediaCMS uwsgi
 
 [Service]
-ExecStart=/home/mediacms.io/bin/uwsgi --ini /home/mediacms.io/mediacms/uwsgi.ini
+ExecStart=/home/mediacms.io/bin/uwsgi --ini /home/mediacms.io/mediacms/deploy/local_install/uwsgi.ini
 ExecStop=/usr/bin/killall -9 uwsgi
 RestartSec=3
 #ExecRestart=killall -9 uwsgi; sleep 5; /home/sss/bin/uwsgi --ini /home/sss/wordgames/uwsgi.ini

deploy/local_install/nginx.conf

@@ -0,0 +1,38 @@
+user www-data;
+worker_processes auto;
+pid /run/nginx.pid;
+
+events {
+    worker_connections 10240;
+}
+
+    worker_rlimit_nofile    20000;  #each connection needs a filehandle (or 2 if you are proxying)
+http {
+    proxy_connect_timeout 75;
+    proxy_read_timeout 12000;
+    client_max_body_size 5800M;
+    sendfile on;
+    tcp_nopush on;
+    tcp_nodelay on;
+    keepalive_timeout 10;
+    types_hash_max_size 2048;
+
+    include /etc/nginx/mime.types;
+    default_type application/octet-stream;
+    
+    access_log /var/log/nginx/access.log;
+    error_log /var/log/nginx/error.log;
+
+    gzip on;
+    gzip_disable "msie6";
+
+    log_format compression '$remote_addr - $remote_user [$time_local] '
+                           '"$request" $status $body_bytes_sent '
+                           '"$http_referer" "$http_user_agent" "$gzip_ratio"';
+
+    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
+
+    include /etc/nginx/conf.d/*.conf;
+    include /etc/nginx/sites-enabled/*;
+}
+

deploy/local_install/uwsgi.ini

@@ -1,8 +1,8 @@
 [uwsgi]
 
-chdir           = /home/mediacms.io/mediacms/
+chdir = /home/mediacms.io/mediacms/
 virtualenv = /home/mediacms.io
-module          = cms.wsgi
+module = cms.wsgi
 
 uid=www-data
 gid=www-data
@@ -10,7 +10,7 @@ gid=www-data
 processes = 2
 threads = 2
 
-master          = true
+master = true
 
 socket = 127.0.0.1:9000
 #socket = /home/mediacms.io/mediacms/deploy/uwsgi.sock
@@ -19,7 +19,7 @@ socket = 127.0.0.1:9000
 workers = 2
 
 
-vacuum          = true
+vacuum = true
 
 logto = /home/mediacms.io/mediacms/logs/errorlog.txt
 

deploy/local_install/uwsgi_params

@@ -0,0 +1,16 @@
+uwsgi_param  QUERY_STRING       $query_string;
+uwsgi_param  REQUEST_METHOD     $request_method;
+uwsgi_param  CONTENT_TYPE       $content_type;
+uwsgi_param  CONTENT_LENGTH     $content_length;
+
+uwsgi_param  REQUEST_URI        $request_uri;
+uwsgi_param  PATH_INFO          $document_uri;
+uwsgi_param  DOCUMENT_ROOT      $document_root;
+uwsgi_param  SERVER_PROTOCOL    $server_protocol;
+uwsgi_param  REQUEST_SCHEME     $scheme;
+uwsgi_param  HTTPS              $https if_not_empty;
+
+uwsgi_param  REMOTE_ADDR        $remote_addr;
+uwsgi_param  REMOTE_PORT        $remote_port;
+uwsgi_param  SERVER_PORT        $server_port;
+uwsgi_param  SERVER_NAME        $server_name;

deploy/uwsgi.ini

@@ -1,19 +0,0 @@
-[uwsgi]
-
-chdir = /home/mediacms.io/mediacms/
-virtualenv = /home/mediacms.io
-module = cms.wsgi
-
-uid = www-data
-gid = www-data
-
-processes = 10
-threads = 10
-master = true
-workers = 8
-vacuum = true
-
-socket = 127.0.0.1:9000
-
-logto = /home/mediacms.io/mediacms/logs/errorlog.txt
-

docker-compose-dev.yaml

@@ -0,0 +1,66 @@
+version: "3"
+
+services:
+  frontend:
+    image: node:14
+    volumes:
+      - ${PWD}/frontend:/home/mediacms.io/mediacms/frontend/
+    working_dir: /home/mediacms.io/mediacms/frontend/
+    command: bash -c "npm install && npm run start"
+    env_file:
+      - ${PWD}/frontend/.env
+    ports:
+      - "8088:8088"
+    depends_on:
+      - web
+  web:
+    build:
+      context: .
+      dockerfile: ./Dockerfile-dev
+    image: mediacms/mediacms-dev:latest
+    ports:
+      - "80:80"
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    depends_on:
+      redis:
+        condition: service_healthy
+      db:
+        condition: service_healthy
+  selenium_hub:
+    container_name: selenium_hub
+    image: selenium/hub
+    ports:
+      - "4444:4444"
+  selenium_chrome:
+    container_name: selenium_chrome
+    image: selenium/node-chrome-debug
+    environment:
+      - HUB_PORT_4444_TCP_ADDR=selenium_hub
+      - HUB_PORT_4444_TCP_PORT=4444
+    ports:
+      - "5900:5900"
+    depends_on:
+      - selenium_hub
+  db:
+    image: postgres
+    volumes:
+      - ../postgres_data:/var/lib/postgresql/data/
+    restart: always
+    environment:
+      POSTGRES_USER: mediacms
+      POSTGRES_PASSWORD: mediacms
+      POSTGRES_DB: mediacms
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U mediacms"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+  redis:
+    image: "redis:alpine"
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3

docker-compose-http-proxy.yaml

@@ -0,0 +1,86 @@
+version: "3"
+
+services:
+  nginx-proxy:
+    image: jwilder/nginx-proxy
+    ports:
+      - "80:80"
+    volumes:
+      - /var/run/docker.sock:/tmp/docker.sock:ro
+      - ./deploy/docker/reverse_proxy/client_max_body_size.conf:/etc/nginx/conf.d/client_max_body_size.conf:ro
+  migrations:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+    depends_on:
+      redis:
+        condition: service_healthy
+      db:
+        condition: service_healthy
+  web:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+      VIRTUAL_HOST: localhost
+    depends_on:
+      - migrations
+  celery_beat:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - redis
+  celery_worker:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  db:
+    image: postgres
+    volumes:
+      - ../postgres_data/:/var/lib/postgresql/data/
+    restart: always
+    environment:
+      POSTGRES_USER: mediacms
+      POSTGRES_PASSWORD: mediacms
+      POSTGRES_DB: mediacms
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U mediacms"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+  redis:
+    image: "redis:alpine"
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli","ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3

docker-compose-https-proxy.yaml

@@ -0,0 +1,88 @@
+version: "3"
+
+services:
+  nginx-proxy:
+    image: jwilder/nginx-proxy
+    ports:
+      - "80:80"
+      - "443:443"
+    volumes:
+      - /var/run/docker.sock:/tmp/docker.sock:ro
+      - ./deploy/docker/reverse_proxy/certs/:/etc/nginx/certs/
+      - ./deploy/docker/reverse_proxy/client_max_body_size.conf:/etc/nginx/conf.d/client_max_body_size.conf:ro
+  migrations:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+    depends_on:
+      redis:
+        condition: service_healthy
+      db:
+        condition: service_healthy
+  web:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+      VIRTUAL_HOST: localhost
+    depends_on:
+      - migrations
+  celery_beat:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - redis
+  celery_worker:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 2
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  db:
+    image: postgres
+    volumes:
+      - ../postgres_data/:/var/lib/postgresql/data/
+    restart: always
+    environment:
+      POSTGRES_USER: mediacms
+      POSTGRES_PASSWORD: mediacms
+      POSTGRES_DB: mediacms
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U mediacms"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+  redis:
+    image: "redis:alpine"
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli","ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3

docker-compose-letsencrypt.yaml

@@ -0,0 +1,115 @@
+version: "3"
+
+# Uses https://github.com/nginx-proxy/acme-companion
+
+services:
+  nginx-proxy:
+    image: nginxproxy/nginx-proxy
+    container_name: nginx-proxy
+    ports:
+      - "80:80"
+      - "443:443"
+    volumes:
+      - conf:/etc/nginx/conf.d
+      - vhost:/etc/nginx/vhost.d
+      - html:/usr/share/nginx/html
+      - dhparam:/etc/nginx/dhparam
+      - certs:/etc/nginx/certs:ro
+      - /var/run/docker.sock:/tmp/docker.sock:ro
+      - ./deploy/docker/reverse_proxy/client_max_body_size.conf:/etc/nginx/conf.d/client_max_body_size.conf:ro
+
+  acme-companion:
+    image: nginxproxy/acme-companion
+    container_name: nginx-proxy-acme
+    volumes_from:
+      - nginx-proxy
+    volumes:
+      - certs:/etc/nginx/certs:rw
+      - acme:/etc/acme.sh
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+
+  migrations:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+    depends_on:
+      redis:
+        condition: service_healthy
+      db:
+        condition: service_healthy
+  web:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+      VIRTUAL_HOST: 'mediacms.52.209.5.113.nip.io'
+      LETSENCRYPT_HOST: 'mediacms.52.209.5.113.nip.io'
+      LETSENCRYPT_EMAIL: 'email@example.com'
+    depends_on:
+      - migrations
+  celery_beat:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - redis
+  celery_worker:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  db:
+    image: postgres
+    volumes:
+      - ../postgres_data:/var/lib/postgresql/data/
+    restart: always
+    environment:
+      POSTGRES_USER: mediacms
+      POSTGRES_PASSWORD: mediacms
+      POSTGRES_DB: mediacms
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U mediacms"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+  redis:
+    image: "redis:alpine"
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli","ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+volumes:
+  conf:
+  vhost:
+  html:
+  dhparam:
+  certs:
+  acme:

docker-compose-named-volumes.yaml

@@ -0,0 +1,88 @@
+version: "3"
+
+services:
+  migrations:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./deploy/docker/local_settings.py:/home/mediacms.io/mediacms/deploy/docker/local_settings.py
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+    depends_on:
+      redis:
+        condition: service_healthy
+      db:
+        condition: service_healthy
+  web:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    ports:
+      - "80:80"
+    volumes:
+      - media_store:/home/mediacms.io/mediacms/media_files/
+      - static_store:/home/mediacms.io/mediacms/static/
+      - ./deploy/docker/local_settings.py:/home/mediacms.io/mediacms/deploy/docker/local_settings.py
+    environment:
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  celery_beat:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./deploy/docker/local_settings.py:/home/mediacms.io/mediacms/deploy/docker/local_settings.py
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - redis
+  celery_worker:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - media_store:/home/mediacms.io/mediacms/media_files/
+      - static_store:/home/mediacms.io/mediacms/static/
+      - ./deploy/docker/local_settings.py:/home/mediacms.io/mediacms/deploy/docker/local_settings.py
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  db:
+    image: postgres
+    volumes:
+      - postgres_data:/var/lib/postgresql/data/
+    restart: always
+    environment:
+      POSTGRES_USER: mediacms
+      POSTGRES_PASSWORD: mediacms
+      POSTGRES_DB: mediacms
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U mediacms"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+  redis:
+    image: "redis:alpine"
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli","ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+volumes:
+  postgres_data:
+  media_store:
+  static_store:

docker-compose.yaml

@@ -0,0 +1,80 @@
+version: "3"
+
+services:
+  migrations:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+    depends_on:
+      redis:
+        condition: service_healthy
+      db:
+        condition: service_healthy
+  web:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    ports:
+      - "80:80"
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  celery_beat:
+    image: mediacms/mediacms:latest
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_SHORT: 'no'
+      ENABLE_CELERY_LONG: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - redis
+  celery_worker:
+    image: mediacms/mediacms:latest
+    deploy:
+      replicas: 1
+    volumes:
+      - ./:/home/mediacms.io/mediacms/
+    environment:
+      ENABLE_UWSGI: 'no'
+      ENABLE_NGINX: 'no'
+      ENABLE_CELERY_BEAT: 'no'
+      ENABLE_MIGRATIONS: 'no'
+    depends_on:
+      - migrations
+  db:
+    image: postgres
+    volumes:
+      - ../postgres_data:/var/lib/postgresql/data/
+    restart: always
+    environment:
+      POSTGRES_USER: mediacms
+      POSTGRES_PASSWORD: mediacms
+      POSTGRES_DB: mediacms
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U mediacms"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+  redis:
+    image: "redis:alpine"
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli","ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3

docs/Add A Static Page To The Sidebar.md

@@ -0,0 +1,133 @@
+## How To Add A Static Page To The Sidebar
+
+### 1. Create your html page in templates/cms/ 
+e.g. duplicate and rename about.html
+```
+sudo cp templates/cms/about.html templates/cms/volunteer.html
+```
+
+### 2. Create your css file in static/css/ 
+```
+touch static/css/volunteer.css
+```
+
+### 3. In your html file, update block headermeta to reflect your new page
+```
+{% block headermeta %}
+<meta property="og:title" content="Volunteer - {{PORTAL_NAME}}">
+<meta property="og:type" content="website">
+<meta property="og:description" content="">
+<meta name="twitter:card" content="summary">
+<script type="application/ld+json">
+{
+	"@context": "https://schema.org",
+	"@type": "BreadcrumbList",
+	"itemListElement": [{
+		"@type": "ListItem",
+		"position": 1,
+        "name": "{{PORTAL_NAME}}",
+        "item": {
+			"@type": "WebPage",
+	        "@id": "{{FRONTEND_HOST}}"
+	    }
+    },
+    {
+		"@type": "ListItem",
+		"position": 2,
+        "name": "Volunteer",
+        "item": {
+			"@type": "VolunteerPage",
+	        "@id": "{{FRONTEND_HOST}}/volunteer"
+	    }
+    }]
+}
+</script>
+<link href="{% static "css/volunteer.css" %}" rel="stylesheet"/>
+{% endblock headermeta %}
+```
+
+### 4. In your html file, update block innercontent to reflect your actual content
+Write whatever you like.
+
+### 5. In your css file, write matching styles for you html file.
+Write whatever you like.
+
+### 6. Add your view to files/views.py
+```
+def volunteer(request):
+    """Volunteer view"""
+    context = {}
+    return render(request, "cms/volunteer.html", context)
+```
+
+### 7. Add your url pattern to files/urls.py
+```
+urlpatterns = [
+    url(r"^$", views.index),
+    url(r"^about", views.about, name="about"),
+    url(r"^volunteer", views.volunteer, name="volunteer"),
+```
+
+### 8. Add your page to the left sidebar
+To add a link to your page as a menu item in the left sidebar,
+add the following code after the last line in _commons.js   
+```
+/* Checks that a given selector has loaded. */
+const checkElement = async selector => {
+    while ( document.querySelector(selector) === null) {
+      await new Promise( resolve =>  requestAnimationFrame(resolve) )
+    }
+    return document.querySelector(selector); 
+  };
+
+/* Checks that sidebar nav menu has loaded, then adds menu item. */
+checkElement('.nav-menu')
+.then((element) => {
+     (function(){    
+        var a = document.createElement('a');        
+        a.href = "/volunteer";
+        a.title = "Volunteer";
+       
+        var s = document.createElement('span');
+        s.className = "menu-item-icon";
+
+        var icon = document.createElement('i');
+        icon.className = "material-icons";
+        icon.setAttribute("data-icon", "people");
+
+        s.appendChild(icon);
+        a.appendChild(s);
+    
+        var linkText = document.createTextNode("Volunteer");
+        var t = document.createElement('span');
+
+        t.appendChild(linkText);
+        a.appendChild(t);
+
+        var listItem = document.createElement('li');
+        listItem.className = "link-item";
+        listItem.appendChild(a);
+
+        //if signed out use 3rd nav-menu
+        var elem = document.querySelector(".nav-menu:nth-child(3) nav ul"); 
+        var loc = elem.innerText;
+        if (loc.includes("About")){
+          elem.insertBefore(listItem, elem.children[2]);
+        } else { //if signed in use 4th nav-menu
+          elem = document.querySelector(".nav-menu:nth-child(4) nav ul");
+          elem.insertBefore(listItem, elem.children[2]);
+        }       
+    })();
+});
+```
+
+### 9. Restart the mediacms web server
+On docker:
+```
+sudo docker stop mediacms_web_1 && sudo docker start mediacms_web_1
+```
+
+Otherwise
+```
+sudo systemctl restart mediacms
+```

docs/Configuration.md

@@ -2,14 +2,25 @@
 
 A number of options are available on `cms/settings.py`. 
 
-It is advisable to override any of them by adding it to `cms/local_settings.py` . 
+It is advisable to override any of them by adding it to `local_settings.py` . 
 
-Any change needs restart of MediaCMS in order to take effect. So edit `cms/local_settings.py`, make a change and restart MediaCMS 
+In case of a the single server installation, add to `cms/local_settings.py` .
 
-```
+In case of a docker compose installation, add to `deploy/docker/local_settings.py` . This will automatically overwrite `cms/local_settings.py` .
+
+Any change needs restart of MediaCMS in order to take effect. 
+
+Single server installation: edit `cms/local_settings.py`, make a change and restart MediaCMS 
+
+```bash
 #systemctl restart mediacms
 ```
 
+Docker Compose installation: edit `deploy/docker/local_settings.py`, make a change and restart MediaCMS containers
+
+```bash
+#docker-compose restart web celery_worker celery_beat
+```
 
 ### change portal logo
 
@@ -94,6 +105,14 @@ Make changes (True/False) to any of the following:
 - CAN_SHARE_MEDIA = True  # whether the share media appears
 ```
 
+### show/hide the download option on a media
+
+Edit `templates/config/installation/features.html` and set 
+
+```
+download: false
+```
+
 ### automatically hide media upon being reported
 
 set a low number for variable `REPORTED_TIMES_THRESHOLD`
@@ -242,3 +261,10 @@ ADMINS_NOTIFICATIONS = {
 - NEW_USER: a new user is added
 - MEDIA_ADDED: a media is added
 - MEDIA_REPORTED: the report for a media was hit
+
+
+
+### Google Analytics
+
+Checkout the instructions by alberto98fx on [Google Analytics](/docs/robots_and_analytics.md) page.
+

docs/Docker_Compose.md

@@ -0,0 +1,52 @@
+# Docker Compose
+
+## Installation
+Install a recent version of [Docker](https://docs.docker.com/get-docker/), and [Docker Compose](https://docs.docker.com/compose/install/).
+
+For Ubuntu 18/20 systems this is:
+
+```bash
+curl -fsSL https://get.docker.com -o get-docker.sh
+sudo sh get-docker.sh
+sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
+sudo chmod +x /usr/local/bin/docker-compose
+```
+
+Then run as root
+
+```bash
+git clone https://github.com/mediacms-io/mediacms
+cd mediacms
+```
+
+The default option is to serve MediaCMS on all ips available of the server (including localhost).
+If you want to explore more options (including setup of https with letsencrypt certificate) checkout the docs on the [Docker deployment](/docs/Docker_deployment.md) page for different docker-compose setups to use.
+
+Run
+
+```bash
+docker-compose up
+```
+
+This will download all MediaCMS related Docker images and start all containers. Once it finishes, MediaCMS will be installed and available on http://localhost or http://ip
+
+
+## Update
+
+Get latest MediaCMS image and stop/start containers
+
+```bash
+cd /path/to/mediacms/installation
+docker pull mediacms/mediacms
+docker-compose down
+docker-compose up
+```
+
+## Configuration
+Checkout the configuration docs on [Configuration](/docs/Configuration.md) page.
+
+
+## Maintenance
+Database is stored on ../postgres_data/ and media_files on media_files/
+
+

docs/Docker_deployment.md

@@ -0,0 +1,53 @@
+# MediaCMS on Docker
+
+The mediacms image is built to use supervisord as the main process, which manages one or more services required to run mediacms. We can toggle which services are run in a given container by setting the environment variables below to `yes` or `no`:
+
+* ENABLE_UWSGI
+* ENABLE_NGINX
+* ENABLE_CELERY_BEAT
+* ENABLE_CELERY_SHORT
+* ENABLE_CELERY_LONG
+* ENABLE_MIGRATIONS
+
+By default, all these services are enabled, but in order to create a scaleable deployment, some of them can be disabled, splitting the service up into smaller services.
+
+Also see the `Dockerfile` for other environment variables which you may wish to override. Application settings, eg. `FRONTEND_HOST` can also be overridden by updating the `deploy/docker/local_settings.py` file.
+
+See example deployments in the sections below. These example deployments have been tested on `docker-compose version 1.27.4` running on `Docker version 19.03.13`
+
+To run, update the configs above if necessary, build the image by running `docker-compose build`, then run `docker-compose run`
+
+# Simple Deployment, accessed as http://localhost
+
+The main container runs migrations, mediacms_web, celery_beat, celery_workers (celery_short and celery_long services), exposed on port 80 supported by redis and postgres database.
+
+ The FRONTEND_HOST in `deploy/docker/local_settings.py` is configured as http://localhost, on the docker host machine.
+
+# Server with ssl certificate through letsencrypt service, accessed as https://my_domain.com
+Before trying this out make sure the ip points to my_domain.com. 
+
+With this method [this deployment](../docker-compose-letsencrypt.yaml) is used. 
+
+Edit this file and set `VIRTUAL_HOST` as my_domain.com, `LETSENCRYPT_HOST` as my_domain.com, and your email on `LETSENCRYPT_EMAIL`
+
+Edit `deploy/docker/local_settings.py` and set https://my_domain.com as `FRONTEND_HOST`
+
+Now run docker-compose -f docker-compose-letsencrypt.yaml up, when installation finishes you will be able to access https://my_domain.com using a valid Letsencrypt certificate!
+
+# Advanced Deployment, accessed as http://localhost:8000
+
+Here we can run 1 mediacms_web instance, with the FRONTEND_HOST in `deploy/docker/local_settings.py` configured as http://localhost:8000. This is bootstrapped by a single migrations instance and supported by a single celery_beat instance and 1 or more celery_worker instances. Redis and postgres containers are also used for persistence. Clients can access the service on http://localhost:8000, on the docker host machine. This is similar to [this deployment](../docker-compose.yaml), with a `port` defined in FRONTEND_HOST.
+
+# Advanced Deployment, with reverse proxy, accessed as http://mediacms.io
+
+Here we can use `jwilder/nginx-proxy` to reverse proxy to 1 or more instances of mediacms_web supported by other services as mentioned in the previous deployment. The FRONTEND_HOST in `deploy/docker/local_settings.py` is configured as http://mediacms.io, nginx-proxy has port 80 exposed. Clients can access the service on http://mediacms.io (Assuming DNS or the hosts file is setup correctly to point to the IP of the nginx-proxy instance). This is similar to [this deployment](../docker-compose-http-proxy.yaml).
+
+# Advanced Deployment, with reverse proxy, accessed as https://localhost
+
+The reverse proxy (`jwilder/nginx-proxy`) can be configured to provide SSL termination using self-signed certificates, letsencrypt or CA signed certificates (see: https://hub.docker.com/r/jwilder/nginx-proxy or [LetsEncrypt Example](https://www.singularaspect.com/use-nginx-proxy-and-letsencrypt-companion-to-host-multiple-websites/) ). In this case the FRONTEND_HOST should be set to https://mediacms.io. This is similar to [this deployment](../docker-compose-http-proxy.yaml).
+
+# A Scaleable Deployment Architecture (Docker, Swarm, Kubernetes)
+
+The architecture below generalises all the deployment scenarios above, and provides a conceptual design for other deployments based on kubernetes and docker swarm. It allows for horizontal scaleability through the use of multiple mediacms_web instances and celery_workers. For large deployments, managed postgres, redis and storage may be adopted.
+
+![MediaCMS](images/architecture.png)

docs/Single_Server.md

@@ -0,0 +1,38 @@
+# Single Server
+
+## Installation
+
+The core dependencies are Python3, Django3, Celery, PostgreSQL, Redis, ffmpeg. Any system that can have these dependencies installed, can run MediaCMS. But we strongly suggest installing on Linux Ubuntu 18 or 20 versions.
+
+Installation on a Ubuntu 18 or 20 system with git utility installed should be completed in a few minutes with the following steps.
+Make sure you run it as user root, on a clear system, since the automatic script will install and configure the following services: Celery/PostgreSQL/Redis/Nginx and will override any existing settings. 
+
+Automated script - tested on Ubuntu 18, Ubuntu 20, and Debian Buster
+
+```bash
+mkdir /home/mediacms.io && cd /home/mediacms.io/
+git clone https://github.com/mediacms-io/mediacms
+cd /home/mediacms.io/mediacms/ && bash ./install.sh
+```
+
+The script will ask if you have a URL where you want to deploy MediaCMS, otherwise it will use localhost. If you provide a URL, it will use Let's Encrypt service to install a valid ssl certificate. 
+
+
+## Update
+
+If you've used the above way to install MediaCMS, update with the following:
+
+```bash
+cd /home/mediacms.io/mediacms # enter mediacms directory
+source  /home/mediacms.io/bin/activate # use virtualenv
+git pull # update code
+python manage.py migrate # run Django migrations
+sudo systemctl restart mediacms celery_long celery_short # restart services
+```
+
+## Configuration
+Checkout the configuration docs on [Configuration](/docs/Configuration.md) page.
+
+
+## Maintenance
+Database can be backed up with pg_dump and media_files on /home/mediacms.io/mediacms/media_files include original files and encoded/transcoded versions

docs/robots_and_analytics.md

@@ -0,0 +1,55 @@
+# Google Analytics
+
+1. Create a file:
+
+``` touch $DIR/mediacms/templates/tracking.html ```
+
+2. Add the Gtag/Analytics script
+
+3. Inside ``` $DIR/mediacms/templates/root.html``` you'll see a file like this one: 
+
+```
+<head>
+    {% block head %}
+
+        <title>{% block headtitle %}{{PORTAL_NAME}}{% endblock headtitle %}</title>
+
+        {% include "common/head-meta.html" %}
+
+        {% block headermeta %}
+        
+        <meta property="og:title" content="{{PORTAL_NAME}}">
+        <meta property="og:type" content="website">
+
+        {%endblock headermeta %}
+
+        {% block externallinks %}{% endblock externallinks %}
+
+        {% include "common/head-links.html" %}
+
+        {% block topimports %}{%endblock topimports %}
+
+        {% include "config/index.html" %}
+      
+    {% endblock head %}
+
+</head>
+```
+
+4. Add  ``` {% include "tracking.html" %} ``` at the end inside the section ```<head>```
+  
+5. If you are using Docker and didn't  mount the entire dir you need to bind a new volume: 
+```
+  
+    web:
+    image: mediacms/mediacms:latest
+    restart: unless-stopped
+    ports:
+      - "80:80"
+    deploy:
+      replicas: 1
+    volumes:
+      - ./templates/root.html:/home/mediacms.io/mediacms/templates/root.html
+      - ./templates/tracking.html://home/mediacms.io/mediacms/templates/tracking.html
+  
+ ```

files/admin.py

@@ -1,14 +1,14 @@
 from django.contrib import admin
 
 from .models import (
-    Media,
-    Encoding,
-    EncodeProfile,
     Category,
     Comment,
-    Tag,
+    EncodeProfile,
+    Encoding,
     Language,
+    Media,
     Subtitle,
+    Tag,
 )
 
 

files/backends.py

@@ -1,9 +1,9 @@
 # ffmpeg only backend
 
-from subprocess import PIPE, Popen
 import locale
-import re
 import logging
+import re
+from subprocess import PIPE, Popen
 
 logger = logging.getLogger(__name__)
 

files/context_processors.py

@@ -1,16 +1,12 @@
 from django.conf import settings
+
 from .methods import is_mediacms_editor, is_mediacms_manager
 
 
 def stuff(request):
     """Pass settings to the frontend"""
     ret = {}
-    if request.is_secure():
-        # in case session is https, pass this setting so
-        # that the frontend uses https too
-        ret["FRONTEND_HOST"] = settings.SSL_FRONTEND_HOST
-    else:
-        ret["FRONTEND_HOST"] = settings.FRONTEND_HOST
+    ret["FRONTEND_HOST"] = request.build_absolute_uri('/')
     ret["DEFAULT_THEME"] = settings.DEFAULT_THEME
     ret["PORTAL_NAME"] = settings.PORTAL_NAME
     ret["LOAD_FROM_CDN"] = settings.LOAD_FROM_CDN
@@ -24,18 +20,12 @@ def stuff(request):
     ret["UPLOAD_MAX_SIZE"] = settings.UPLOAD_MAX_SIZE
     ret["UPLOAD_MAX_FILES_NUMBER"] = settings.UPLOAD_MAX_FILES_NUMBER
     ret["PRE_UPLOAD_MEDIA_MESSAGE"] = settings.PRE_UPLOAD_MEDIA_MESSAGE
-    ret[
-        "POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY"
-    ] = settings.POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY
+    ret["POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY"] = settings.POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY
     ret["IS_MEDIACMS_ADMIN"] = request.user.is_superuser
     ret["IS_MEDIACMS_EDITOR"] = is_mediacms_editor(request.user)
     ret["IS_MEDIACMS_MANAGER"] = is_mediacms_manager(request.user)
     ret["ALLOW_RATINGS"] = settings.ALLOW_RATINGS
-    ret[
-        "ALLOW_RATINGS_CONFIRMED_EMAIL_ONLY"
-    ] = settings.ALLOW_RATINGS_CONFIRMED_EMAIL_ONLY
-    ret[
-        "VIDEO_PLAYER_FEATURED_VIDEO_ON_INDEX_PAGE"
-    ] = settings.VIDEO_PLAYER_FEATURED_VIDEO_ON_INDEX_PAGE
+    ret["ALLOW_RATINGS_CONFIRMED_EMAIL_ONLY"] = settings.ALLOW_RATINGS_CONFIRMED_EMAIL_ONLY
+    ret["VIDEO_PLAYER_FEATURED_VIDEO_ON_INDEX_PAGE"] = settings.VIDEO_PLAYER_FEATURED_VIDEO_ON_INDEX_PAGE
     ret["RSS_URL"] = "/rss"
     return ret

files/feeds.py

@@ -1,12 +1,11 @@
-from django.contrib.syndication.views import Feed
-from django.utils.feedgenerator import Rss201rev2Feed
-from django.urls import reverse
-from django.db.models import Q
 from django.conf import settings
 from django.contrib.postgres.search import SearchQuery
+from django.contrib.syndication.views import Feed
+from django.urls import reverse
+from django.utils.feedgenerator import Rss201rev2Feed
 
-from .models import Media, Category
 from . import helpers
+from .models import Media
 from .stop_words import STOP_WORDS
 
 
@@ -119,11 +118,7 @@ class SearchRSSFeed(Feed):
         elif query:
             # same as on files.views.MediaSearch: move this processing to a prepare_query function
             query = helpers.clean_query(query)
-            q_parts = [
-                q_part.rstrip("y")
-                for q_part in query.split()
-                if q_part not in STOP_WORDS
-            ]
+            q_parts = [q_part.rstrip("y") for q_part in query.split() if q_part not in STOP_WORDS]
             if q_parts:
                 query = SearchQuery(q_parts[0] + ":*", search_type="raw")
                 for part in q_parts[1:]:

files/forms.py

@@ -1,6 +1,7 @@
 from django import forms
+
+from .methods import get_next_state, is_mediacms_editor
 from .models import Media, Subtitle
-from .methods import is_mediacms_editor, get_next_state
 
 
 class MultipleSelect(forms.CheckboxSelectMultiple):
@@ -8,9 +9,7 @@ class MultipleSelect(forms.CheckboxSelectMultiple):
 
 
 class MediaForm(forms.ModelForm):
-    new_tags = forms.CharField(
-        label="Tags", help_text="a comma separated list of new tags.", required=False
-    )
+    new_tags = forms.CharField(label="Tags", help_text="a comma separated list of new tags.", required=False)
 
     class Meta:
         model = Media
@@ -27,7 +26,7 @@ class MediaForm(forms.ModelForm):
             "thumbnail_time",
             "reported_times",
             "is_reviewed",
-            "allow_download"
+            "allow_download",
         )
         widgets = {
             "tags": MultipleSelect(),
@@ -42,9 +41,7 @@ class MediaForm(forms.ModelForm):
             self.fields.pop("featured")
             self.fields.pop("reported_times")
             self.fields.pop("is_reviewed")
-        self.fields["new_tags"].initial = ", ".join(
-            [tag.title for tag in self.instance.tags.all()]
-        )
+        self.fields["new_tags"].initial = ", ".join([tag.title for tag in self.instance.tags.all()])
 
     def clean_uploaded_poster(self):
         image = self.cleaned_data.get("uploaded_poster", False)
@@ -57,9 +54,7 @@ class MediaForm(forms.ModelForm):
         data = self.cleaned_data
         state = data.get("state")
         if state != self.initial["state"]:
-            self.instance.state = get_next_state(
-                self.user, self.initial["state"], self.instance.state
-            )
+            self.instance.state = get_next_state(self.user, self.initial["state"], self.instance.state)
 
         media = super(MediaForm, self).save(*args, **kwargs)
         return media

files/helpers.py

@@ -1,19 +1,19 @@
 # Kudos to Werner Robitza, AVEQ GmbH, for helping with ffmpeg
 # related content
 
-import os
-import math
-import shutil
-import tempfile
-import random
 import hashlib
-import subprocess
 import json
+import math
+import os
+import random
+import shutil
+import subprocess
+import tempfile
 from fractions import Fraction
+
 import filetype
 from django.conf import settings
 
-
 CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
 
 CRF_ENCODING_NUM_SECONDS = 2  # 0 * 60 # videos with greater duration will get
@@ -168,9 +168,7 @@ def rm_dir(directory):
 
 def url_from_path(filename):
     # TODO: find a way to preserver http - https ...
-    return "{0}{1}".format(
-        settings.MEDIA_URL, filename.replace(settings.MEDIA_ROOT, "")
-    )
+    return "{0}{1}".format(settings.MEDIA_URL, filename.replace(settings.MEDIA_ROOT, ""))
 
 
 def create_temp_file(suffix=None, dir=settings.TEMP_DIRECTORY):
@@ -210,9 +208,7 @@ def run_command(cmd, cwd=None):
         cmd = cmd.split()
     ret = {}
     if cwd:
-        process = subprocess.Popen(
-            cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd
-        )
+        process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd)
     else:
         process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     stdout, stderr = process.communicate()
@@ -331,9 +327,7 @@ def media_file_info(input_file):
         except ValueError:
             hms, msec = duration_str.split(",")
 
-        total_dur = sum(
-            int(x) * 60 ** i for i, x in enumerate(reversed(hms.split(":")))
-        )
+        total_dur = sum(int(x) * 60 ** i for i, x in enumerate(reversed(hms.split(":"))))
         video_duration = total_dur + float("0." + msec)
     else:
         # fallback to format, eg for webm
@@ -370,7 +364,7 @@ def media_file_info(input_file):
             input_file,
         ]
         stdout = run_command(cmd).get("out")
-        stream_size = sum([int(l) for l in stdout.split("\n") if l != ""])
+        stream_size = sum([int(line) for line in stdout.split("\n") if line != ""])
         video_bitrate = round((stream_size * 8 / 1024.0) / video_duration, 2)
 
     ret = {
@@ -387,7 +381,6 @@ def media_file_info(input_file):
     }
 
     if has_audio:
-        audio_duration = 1
         if "duration" in audio_info.keys():
             audio_duration = float(audio_info["duration"])
         elif "tags" in audio_info.keys() and "DURATION" in audio_info["tags"]:
@@ -396,9 +389,7 @@ def media_file_info(input_file):
                 hms, msec = duration_str.split(".")
             except ValueError:
                 hms, msec = duration_str.split(",")
-            total_dur = sum(
-                int(x) * 60 ** i for i, x in enumerate(reversed(hms.split(":")))
-            )
+            total_dur = sum(int(x) * 60 ** i for i, x in enumerate(reversed(hms.split(":"))))
             audio_duration = total_dur + float("0." + msec)
         else:
             # fallback to format, eg for webm
@@ -432,7 +423,7 @@ def media_file_info(input_file):
                 input_file,
             ]
             stdout = run_command(cmd).get("out")
-            stream_size = sum([int(l) for l in stdout.split("\n") if l != ""])
+            stream_size = sum([int(line) for line in stdout.split("\n") if line != ""])
             audio_bitrate = round((stream_size * 8 / 1024.0) / audio_duration, 2)
 
         ret.update(
@@ -660,9 +651,7 @@ def get_base_ffmpeg_command(
     return cmd
 
 
-def produce_ffmpeg_commands(
-    media_file, media_info, resolution, codec, output_filename, pass_file, chunk=False
-):
+def produce_ffmpeg_commands(media_file, media_info, resolution, codec, output_filename, pass_file, chunk=False):
     try:
         media_info = json.loads(media_info)
     except BaseException:
@@ -670,13 +659,13 @@ def produce_ffmpeg_commands(
 
     if codec == "h264":
         encoder = "libx264"
-        ext = "mp4"
+        # ext = "mp4"
     elif codec in ["h265", "hevc"]:
         encoder = "libx265"
-        ext = "mp4"
+        # ext = "mp4"
     elif codec == "vp9":
         encoder = "libvpx-vp9"
-        ext = "webm"
+        # ext = "webm"
     else:
         return False
 
@@ -699,9 +688,7 @@ def produce_ffmpeg_commands(
     #    else:
 
     # adjust the target frame rate if the input is fractional
-    target_fps = (
-        src_framerate if isinstance(src_framerate, int) else math.ceil(src_framerate)
-    )
+    target_fps = src_framerate if isinstance(src_framerate, int) else math.ceil(src_framerate)
 
     if media_info.get("video_duration") > CRF_ENCODING_NUM_SECONDS:
         enc_type = "crf"

files/management_views.py

@@ -1,16 +1,18 @@
-from rest_framework.views import APIView
-from rest_framework.parsers import JSONParser
-from rest_framework.settings import api_settings
-from rest_framework.response import Response
+from drf_yasg import openapi as openapi
+from drf_yasg.utils import swagger_auto_schema
 from rest_framework import status
+from rest_framework.parsers import JSONParser
+from rest_framework.response import Response
+from rest_framework.settings import api_settings
+from rest_framework.views import APIView
 
 from users.models import User
 from users.serializers import UserSerializer
-from .permissions import IsMediacmsEditor
-from .models import Media, Comment
-from .methods import is_mediacms_manager
 
-from .serializers import MediaSerializer, CommentSerializer
+from .methods import is_mediacms_manager
+from .models import Comment, Media
+from .permissions import IsMediacmsEditor
+from .serializers import CommentSerializer, MediaSerializer
 
 
 class MediaList(APIView):
@@ -23,6 +25,17 @@ class MediaList(APIView):
     permission_classes = (IsMediacmsEditor,)
     parser_classes = (JSONParser,)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='sort_by', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='Sort by any of: title, add_date, edit_date, views, likes, reported_times'),
+            openapi.Parameter(name='ordering', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='Order by: asc, desc'),
+            openapi.Parameter(name='state', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='Media state, options: private", "public", "unlisted'),
+            openapi.Parameter(name='encoding_status', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='Encoding status, options "pending", "running", "fail", "success"'),
+        ],
+        tags=['Manage'],
+        operation_summary='Manage Media',
+        operation_description='Manage media for MediaCMS managers and reviewers',
+    )
     def get(self, request, format=None):
         params = self.request.query_params
         ordering = params.get("ordering", "").strip()
@@ -94,6 +107,12 @@ class MediaList(APIView):
         serializer = MediaSerializer(page, many=True, context={"request": request})
         return paginator.get_paginated_response(serializer.data)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Manage'],
+        operation_summary='Delete Media',
+        operation_description='Delete media for MediaCMS managers and reviewers',
+    )
     def delete(self, request, format=None):
         tokens = request.GET.get("tokens")
         if tokens:
@@ -112,6 +131,12 @@ class CommentList(APIView):
     permission_classes = (IsMediacmsEditor,)
     parser_classes = (JSONParser,)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Manage'],
+        operation_summary='Manage Comments',
+        operation_description='Manage comments for MediaCMS managers and reviewers',
+    )
     def get(self, request, format=None):
         params = self.request.query_params
         ordering = params.get("ordering", "").strip()
@@ -137,6 +162,12 @@ class CommentList(APIView):
         serializer = CommentSerializer(page, many=True, context={"request": request})
         return paginator.get_paginated_response(serializer.data)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Manage'],
+        operation_summary='Delete Comments',
+        operation_description='Delete comments for MediaCMS managers and reviewers',
+    )
     def delete(self, request, format=None):
         comment_ids = request.GET.get("comment_ids")
         if comment_ids:
@@ -156,6 +187,12 @@ class UserList(APIView):
     permission_classes = (IsMediacmsEditor,)
     parser_classes = (JSONParser,)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Manage'],
+        operation_summary='Manage Users',
+        operation_description='Manage users for MediaCMS managers and reviewers',
+    )
     def get(self, request, format=None):
         params = self.request.query_params
         ordering = params.get("ordering", "").strip()
@@ -187,11 +224,15 @@ class UserList(APIView):
         serializer = UserSerializer(page, many=True, context={"request": request})
         return paginator.get_paginated_response(serializer.data)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Manage'],
+        operation_summary='Delete Users',
+        operation_description='Delete users for MediaCMS managers',
+    )
     def delete(self, request, format=None):
         if not is_mediacms_manager(request.user):
-            return Response(
-                {"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST)
 
         tokens = request.GET.get("tokens")
         if tokens:

files/methods.py

@@ -1,15 +1,17 @@
 # Kudos to Werner Robitza, AVEQ GmbH, for helping with ffmpeg
 # related content
 
+import itertools
 import logging
 import random
-import itertools
 from datetime import datetime
-from cms import celery_app
+
 from django.conf import settings
 from django.core.cache import cache
-from django.db.models import Q
 from django.core.mail import EmailMessage
+from django.db.models import Q
+
+from cms import celery_app
 
 from . import models
 from .helpers import mask_ip
@@ -48,9 +50,7 @@ def pre_save_action(media, user, session_key, action, remote_ip):
     if user:
         query = MediaAction.objects.filter(media=media, action=action, user=user)
     else:
-        query = MediaAction.objects.filter(
-            media=media, action=action, session_key=session_key
-        )
+        query = MediaAction.objects.filter(media=media, action=action, session_key=session_key)
     query = query.order_by("-action_date")
 
     if query:
@@ -71,11 +71,7 @@ def pre_save_action(media, user, session_key, action, remote_ip):
         # perform some checking for requests where no session
         # id is specified (and user is anonymous) to avoid spam
         # eg allow for the same remote_ip for a specific number of actions
-        query = (
-            MediaAction.objects.filter(media=media, action=action, remote_ip=remote_ip)
-            .filter(user=None)
-            .order_by("-action_date")
-        )
+        query = MediaAction.objects.filter(media=media, action=action, remote_ip=remote_ip).filter(user=None).order_by("-action_date")
         if query:
             query = query.first()
             now = datetime.now(query.action_date.tzinfo)
@@ -204,11 +200,8 @@ URL: %s
             d["to"] = [media.user.email]
             notify_items.append(d)
 
-
     for item in notify_items:
-        email = EmailMessage(
-            item["title"], item["msg"], settings.DEFAULT_FROM_EMAIL, item["to"]
-        )
+        email = EmailMessage(item["title"], item["msg"], settings.DEFAULT_FROM_EMAIL, item["to"])
         email.send(fail_silently=True)
     return True
 
@@ -222,17 +215,9 @@ def show_recommended_media(request, limit=100):
     pmi = cache.get("popular_media_ids")
     # produced by task get_list_of_popular_media and cached
     if pmi:
-        media = list(
-            models.Media.objects.filter(friendly_token__in=pmi)
-            .filter(basic_query)
-            .prefetch_related("user")[:limit]
-        )
+        media = list(models.Media.objects.filter(friendly_token__in=pmi).filter(basic_query).prefetch_related("user")[:limit])
     else:
-        media = list(
-            models.Media.objects.filter(basic_query)
-            .order_by("-views", "-likes")
-            .prefetch_related("user")[:limit]
-        )
+        media = list(models.Media.objects.filter(basic_query).order_by("-views", "-likes").prefetch_related("user")[:limit])
     random.shuffle(media)
     return media
 
@@ -257,11 +242,7 @@ def show_related_media_content(media, request, limit):
     # and include author videos in any case
 
     q_author = Q(listable=True, user=media.user)
-    m = list(
-        models.Media.objects.filter(q_author)
-        .order_by()
-        .prefetch_related("user")[:limit]
-    )
+    m = list(models.Media.objects.filter(q_author).order_by().prefetch_related("user")[:limit])
 
     # order by random criteria so that it doesn't bring the same results
     # attention: only fields that are indexed make sense here! also need
@@ -282,20 +263,12 @@ def show_related_media_content(media, request, limit):
         category = media.category.first()
         if category:
             q_category = Q(listable=True, category=category)
-            q_res = (
-                models.Media.objects.filter(q_category)
-                .order_by(order_criteria[random.randint(0, len(order_criteria) - 1)])
-                .prefetch_related("user")[: limit - media.user.media_count]
-            )
+            q_res = models.Media.objects.filter(q_category).order_by(order_criteria[random.randint(0, len(order_criteria) - 1)]).prefetch_related("user")[: limit - media.user.media_count]
             m = list(itertools.chain(m, q_res))
 
         if len(m) < limit:
             q_generic = Q(listable=True)
-            q_res = (
-                models.Media.objects.filter(q_generic)
-                .order_by(order_criteria[random.randint(0, len(order_criteria) - 1)])
-                .prefetch_related("user")[: limit - media.user.media_count]
-            )
+            q_res = models.Media.objects.filter(q_generic).order_by(order_criteria[random.randint(0, len(order_criteria) - 1)]).prefetch_related("user")[: limit - media.user.media_count]
             m = list(itertools.chain(m, q_res))
 
     m = list(set(m[:limit]))  # remove duplicates
@@ -313,11 +286,7 @@ def show_related_media_author(media, request, limit):
     """Return a list of related media form the same author"""
 
     q_author = Q(listable=True, user=media.user)
-    m = list(
-        models.Media.objects.filter(q_author)
-        .order_by()
-        .prefetch_related("user")[:limit]
-    )
+    m = list(models.Media.objects.filter(q_author).order_by().prefetch_related("user")[:limit])
 
     # order by random criteria so that it doesn't bring the same results
     # attention: only fields that are indexed make sense here! also need
@@ -347,13 +316,7 @@ def update_user_ratings(user, media, user_ratings):
     """Populate user ratings for a media"""
 
     for rating in user_ratings:
-        user_rating = (
-            models.Rating.objects.filter(
-                user=user, media_id=media, rating_category_id=rating.get("category_id")
-            )
-            .only("score")
-            .first()
-        )
+        user_rating = models.Rating.objects.filter(user=user, media_id=media, rating_category_id=rating.get("category_id")).only("score").first()
         if user_rating:
             rating["score"] = user_rating.score
     return user_ratings
@@ -379,9 +342,7 @@ View it on %s
             media.title,
             media_url,
         )
-        email = EmailMessage(
-            title, msg, settings.DEFAULT_FROM_EMAIL, [media.user.email]
-        )
+        email = EmailMessage(title, msg, settings.DEFAULT_FROM_EMAIL, [media.user.email])
         email.send(fail_silently=True)
     return True
 
@@ -420,27 +381,17 @@ def list_tasks():
                     friendly_token = task_args.split()[0]
                     profile_id = task_args.split()[1]
 
-                    media = models.Media.objects.filter(
-                        friendly_token=friendly_token
-                    ).first()
+                    media = models.Media.objects.filter(friendly_token=friendly_token).first()
                     if media:
-                        profile = models.EncodeProfile.objects.filter(
-                            id=profile_id
-                        ).first()
+                        profile = models.EncodeProfile.objects.filter(id=profile_id).first()
                         if profile:
-                            media_profile_pairs.append(
-                                (media.friendly_token, profile.id)
-                            )
+                            media_profile_pairs.append((media.friendly_token, profile.id))
                             task_dict["info"] = {}
                             task_dict["info"]["profile name"] = profile.name
                             task_dict["info"]["media title"] = media.title
-                            encoding = models.Encoding.objects.filter(
-                                task_id=task.get("id")
-                            ).first()
+                            encoding = models.Encoding.objects.filter(task_id=task.get("id")).first()
                             if encoding:
-                                task_dict["info"][
-                                    "encoding progress"
-                                ] = encoding.progress
+                                task_dict["info"]["encoding progress"] = encoding.progress
 
                 ret[state]["tasks"].append(task_dict)
     ret["task_ids"] = task_ids

files/migrations/0001_initial.py

@@ -1,11 +1,13 @@
 # Generated by Django 3.1.4 on 2020-12-01 07:12
 
-import django.contrib.postgres.search
-from django.db import migrations, models
-import files.models
-import imagekit.models.fields
 import uuid
 
+import django.contrib.postgres.search
+import imagekit.models.fields
+from django.db import migrations, models
+
+import files.models
+
 
 class Migration(migrations.Migration):
 
@@ -32,9 +34,7 @@ class Migration(migrations.Migration):
                 ("description", models.TextField(blank=True)),
                 (
                     "is_global",
-                    models.BooleanField(
-                        default=False, help_text="global categories or user specific"
-                    ),
+                    models.BooleanField(default=False, help_text="global categories or user specific"),
                 ),
                 (
                     "media_count",
@@ -42,9 +42,7 @@ class Migration(migrations.Migration):
                 ),
                 (
                     "thumbnail",
-                    imagekit.models.fields.ProcessedImageField(
-                        blank=True, upload_to=files.models.category_thumb_path
-                    ),
+                    imagekit.models.fields.ProcessedImageField(blank=True, upload_to=files.models.category_thumb_path),
                 ),
                 (
                     "listings_thumbnail",
@@ -153,9 +151,7 @@ class Migration(migrations.Migration):
                 ("commands", models.TextField(blank=True, help_text="commands run")),
                 (
                     "chunk",
-                    models.BooleanField(
-                        db_index=True, default=False, help_text="is chunk?"
-                    ),
+                    models.BooleanField(db_index=True, default=False, help_text="is chunk?"),
                 ),
                 ("chunk_file_path", models.CharField(blank=True, max_length=400)),
                 ("chunks_info", models.TextField(blank=True)),
@@ -317,9 +313,7 @@ class Migration(migrations.Migration):
                 ("likes", models.IntegerField(db_index=True, default=1)),
                 (
                     "listable",
-                    models.BooleanField(
-                        default=False, help_text="Whether it will appear on listings"
-                    ),
+                    models.BooleanField(default=False, help_text="Whether it will appear on listings"),
                 ),
                 (
                     "md5sum",
@@ -341,9 +335,7 @@ class Migration(migrations.Migration):
                 ),
                 (
                     "media_info",
-                    models.TextField(
-                        blank=True, help_text="extracted media metadata info"
-                    ),
+                    models.TextField(blank=True, help_text="extracted media metadata info"),
                 ),
                 (
                     "media_type",
@@ -387,9 +379,7 @@ class Migration(migrations.Migration):
                 ),
                 (
                     "reported_times",
-                    models.IntegerField(
-                        default=0, help_text="how many time a Medis is reported"
-                    ),
+                    models.IntegerField(default=0, help_text="how many time a Medis is reported"),
                 ),
                 (
                     "search",
@@ -485,9 +475,7 @@ class Migration(migrations.Migration):
                 ),
                 (
                     "user_featured",
-                    models.BooleanField(
-                        default=False, help_text="Featured by the user"
-                    ),
+                    models.BooleanField(default=False, help_text="Featured by the user"),
                 ),
                 ("video_height", models.IntegerField(default=1)),
                 ("views", models.IntegerField(db_index=True, default=1)),

files/migrations/0002_auto_20201201_0712.py

@@ -1,10 +1,10 @@
 # Generated by Django 3.1.4 on 2020-12-01 07:12
 
-from django.conf import settings
 import django.contrib.postgres.indexes
-from django.db import migrations, models
 import django.db.models.deletion
 import mptt.fields
+from django.conf import settings
+from django.db import migrations, models
 
 
 class Migration(migrations.Migration):
@@ -31,9 +31,7 @@ class Migration(migrations.Migration):
         migrations.AddField(
             model_name="subtitle",
             name="language",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to="files.language"
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="files.language"),
         ),
         migrations.AddField(
             model_name="subtitle",
@@ -47,9 +45,7 @@ class Migration(migrations.Migration):
         migrations.AddField(
             model_name="subtitle",
             name="user",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
         ),
         migrations.AddField(
             model_name="rating",
@@ -63,37 +59,27 @@ class Migration(migrations.Migration):
         migrations.AddField(
             model_name="rating",
             name="rating_category",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to="files.ratingcategory"
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="files.ratingcategory"),
         ),
         migrations.AddField(
             model_name="rating",
             name="user",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
         ),
         migrations.AddField(
             model_name="playlistmedia",
             name="media",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to="files.media"
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="files.media"),
         ),
         migrations.AddField(
             model_name="playlistmedia",
             name="playlist",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to="files.playlist"
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="files.playlist"),
         ),
         migrations.AddField(
             model_name="playlist",
             name="media",
-            field=models.ManyToManyField(
-                blank=True, through="files.PlaylistMedia", to="files.Media"
-            ),
+            field=models.ManyToManyField(blank=True, through="files.PlaylistMedia", to="files.Media"),
         ),
         migrations.AddField(
             model_name="playlist",
@@ -173,9 +159,7 @@ class Migration(migrations.Migration):
         migrations.AddField(
             model_name="encoding",
             name="profile",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to="files.encodeprofile"
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="files.encodeprofile"),
         ),
         migrations.AddField(
             model_name="comment",
@@ -200,9 +184,7 @@ class Migration(migrations.Migration):
         migrations.AddField(
             model_name="comment",
             name="user",
-            field=models.ForeignKey(
-                on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL
-            ),
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
         ),
         migrations.AddField(
             model_name="category",
@@ -216,9 +198,7 @@ class Migration(migrations.Migration):
         ),
         migrations.AddIndex(
             model_name="rating",
-            index=models.Index(
-                fields=["user", "media"], name="files_ratin_user_id_72ca6a_idx"
-            ),
+            index=models.Index(fields=["user", "media"], name="files_ratin_user_id_72ca6a_idx"),
         ),
         migrations.AlterUniqueTogether(
             name="rating",
@@ -226,8 +206,6 @@ class Migration(migrations.Migration):
         ),
         migrations.AddIndex(
             model_name="media",
-            index=django.contrib.postgres.indexes.GinIndex(
-                fields=["search"], name="files_media_search_7194c6_gin"
-            ),
+            index=django.contrib.postgres.indexes.GinIndex(fields=["search"], name="files_media_search_7194c6_gin"),
         ),
     ]

files/models.py

@@ -1,31 +1,29 @@
+import json
 import logging
-import uuid
 import os
+import random
 import re
 import tempfile
-import random
-import json
+import uuid
+
 import m3u8
-from django.utils import timezone
-from django.db import connection
-from django.db import models
-from django.template.defaultfilters import slugify
 from django.conf import settings
 from django.contrib.postgres.indexes import GinIndex
-from django.db.models.signals import pre_delete, post_delete, post_save, m2m_changed
-from django.core.files import File
-from django.core.exceptions import ValidationError
-from django.dispatch import receiver
-from django.urls import reverse
-from django.utils.html import strip_tags
 from django.contrib.postgres.search import SearchVectorField
+from django.core.exceptions import ValidationError
+from django.core.files import File
+from django.db import connection, models
+from django.db.models.signals import m2m_changed, post_delete, post_save, pre_delete
+from django.dispatch import receiver
+from django.template.defaultfilters import slugify
+from django.urls import reverse
+from django.utils import timezone
+from django.utils.html import strip_tags
+from imagekit.models import ProcessedImageField
+from imagekit.processors import ResizeToFit
 from mptt.models import MPTTModel, TreeForeignKey
 
-from imagekit.processors import ResizeToFit
-from imagekit.models import ProcessedImageField
-
 from . import helpers
-from .methods import notify_users
 from .stop_words import STOP_WORDS
 
 logger = logging.getLogger(__name__)
@@ -88,36 +86,26 @@ ENCODE_RESOLUTIONS_KEYS = [resolution for resolution, name in ENCODE_RESOLUTIONS
 def original_media_file_path(instance, filename):
     """Helper function to place original media file"""
     file_name = "{0}.{1}".format(instance.uid.hex, helpers.get_file_name(filename))
-    return settings.MEDIA_UPLOAD_DIR + "user/{0}/{1}".format(
-        instance.user.username, file_name
-    )
+    return settings.MEDIA_UPLOAD_DIR + "user/{0}/{1}".format(instance.user.username, file_name)
 
 
 def encoding_media_file_path(instance, filename):
     """Helper function to place encoded media file"""
 
-    file_name = "{0}.{1}".format(
-        instance.media.uid.hex, helpers.get_file_name(filename)
-    )
-    return settings.MEDIA_ENCODING_DIR + "{0}/{1}/{2}".format(
-        instance.profile.id, instance.media.user.username, file_name
-    )
+    file_name = "{0}.{1}".format(instance.media.uid.hex, helpers.get_file_name(filename))
+    return settings.MEDIA_ENCODING_DIR + "{0}/{1}/{2}".format(instance.profile.id, instance.media.user.username, file_name)
 
 
 def original_thumbnail_file_path(instance, filename):
     """Helper function to place original media thumbnail file"""
 
-    return settings.THUMBNAIL_UPLOAD_DIR + "user/{0}/{1}".format(
-        instance.user.username, filename
-    )
+    return settings.THUMBNAIL_UPLOAD_DIR + "user/{0}/{1}".format(instance.user.username, filename)
 
 
 def subtitles_file_path(instance, filename):
     """Helper function to place subtitle file"""
 
-    return settings.SUBTITLES_UPLOAD_DIR + "user/{0}/{1}".format(
-        instance.media.user.username, filename
-    )
+    return settings.SUBTITLES_UPLOAD_DIR + "user/{0}/{1}".format(instance.media.user.username, filename)
 
 
 def category_thumb_path(instance, filename):
@@ -130,17 +118,11 @@ def category_thumb_path(instance, filename):
 class Media(models.Model):
     """The most important model for MediaCMS"""
 
-    add_date = models.DateTimeField(
-        "Date produced", blank=True, null=True, db_index=True
-    )
+    add_date = models.DateTimeField("Date produced", blank=True, null=True, db_index=True)
 
-    allow_download = models.BooleanField(
-        default=True, help_text="Whether option to download media is shown"
-    )
+    allow_download = models.BooleanField(default=True, help_text="Whether option to download media is shown")
 
-    category = models.ManyToManyField(
-        "Category", blank=True, help_text="Media can be part of one or more categories"
-    )
+    category = models.ManyToManyField("Category", blank=True, help_text="Media can be part of one or more categories")
 
     channel = models.ForeignKey(
         "users.Channel",
@@ -158,13 +140,9 @@ class Media(models.Model):
 
     edit_date = models.DateTimeField(auto_now=True)
 
-    enable_comments = models.BooleanField(
-        default=True, help_text="Whether comments will be allowed for this media"
-    )
+    enable_comments = models.BooleanField(default=True, help_text="Whether comments will be allowed for this media")
 
-    encoding_status = models.CharField(
-        max_length=20, choices=MEDIA_ENCODING_STATUS, default="pending", db_index=True
-    )
+    encoding_status = models.CharField(max_length=20, choices=MEDIA_ENCODING_STATUS, default="pending", db_index=True)
 
     featured = models.BooleanField(
         default=False,
@@ -172,13 +150,9 @@ class Media(models.Model):
         help_text="Whether media is globally featured by a MediaCMS editor",
     )
 
-    friendly_token = models.CharField(
-        blank=True, max_length=12, db_index=True, help_text="Identifier for the Media"
-    )
+    friendly_token = models.CharField(blank=True, max_length=12, db_index=True, help_text="Identifier for the Media")
 
-    hls_file = models.CharField(
-        max_length=1000, blank=True, help_text="Path to HLS file for videos"
-    )
+    hls_file = models.CharField(max_length=1000, blank=True, help_text="Path to HLS file for videos")
 
     is_reviewed = models.BooleanField(
         default=settings.MEDIA_IS_REVIEWED,
@@ -186,19 +160,13 @@ class Media(models.Model):
         help_text="Whether media is reviewed, so it can appear on public listings",
     )
 
-    license = models.ForeignKey(
-        "License", on_delete=models.CASCADE, db_index=True, blank=True, null=True
-    )
+    license = models.ForeignKey("License", on_delete=models.CASCADE, db_index=True, blank=True, null=True)
 
     likes = models.IntegerField(db_index=True, default=1)
 
-    listable = models.BooleanField(
-        default=False, help_text="Whether it will appear on listings"
-    )
+    listable = models.BooleanField(default=False, help_text="Whether it will appear on listings")
 
-    md5sum = models.CharField(
-        max_length=50, blank=True, null=True, help_text="Not exposed, used internally"
-    )
+    md5sum = models.CharField(max_length=50, blank=True, null=True, help_text="Not exposed, used internally")
 
     media_file = models.FileField(
         "media file",
@@ -217,9 +185,7 @@ class Media(models.Model):
         default="video",
     )
 
-    password = models.CharField(
-        max_length=100, blank=True, help_text="password for private media"
-    )
+    password = models.CharField(max_length=100, blank=True, help_text="password for private media")
 
     preview_file_path = models.CharField(
         max_length=500,
@@ -243,9 +209,7 @@ class Media(models.Model):
         help_text="Rating category, if media Rating is allowed",
     )
 
-    reported_times = models.IntegerField(
-        default=0, help_text="how many time a Medis is reported"
-    )
+    reported_times = models.IntegerField(default=0, help_text="how many time a Medis is reported")
 
     search = SearchVectorField(
         null=True,
@@ -274,13 +238,9 @@ class Media(models.Model):
         help_text="state of Media",
     )
 
-    tags = models.ManyToManyField(
-        "Tag", blank=True, help_text="select one or more out of the existing tags"
-    )
+    tags = models.ManyToManyField("Tag", blank=True, help_text="select one or more out of the existing tags")
 
-    title = models.CharField(
-        max_length=100, help_text="media title", blank=True, db_index=True
-    )
+    title = models.CharField(max_length=100, help_text="media title", blank=True, db_index=True)
 
     thumbnail = ProcessedImageField(
         upload_to=original_thumbnail_file_path,
@@ -292,13 +252,9 @@ class Media(models.Model):
         help_text="media extracted small thumbnail, shown on listings",
     )
 
-    thumbnail_time = models.FloatField(
-        blank=True, null=True, help_text="Time on video that a thumbnail will be taken"
-    )
+    thumbnail_time = models.FloatField(blank=True, null=True, help_text="Time on video that a thumbnail will be taken")
 
-    uid = models.UUIDField(
-        unique=True, default=uuid.uuid4, help_text="A unique identifier for the Media"
-    )
+    uid = models.UUIDField(unique=True, default=uuid.uuid4, help_text="A unique identifier for the Media")
 
     uploaded_thumbnail = ProcessedImageField(
         upload_to=original_thumbnail_file_path,
@@ -321,9 +277,7 @@ class Media(models.Model):
         max_length=500,
     )
 
-    user = models.ForeignKey(
-        "users.User", on_delete=models.CASCADE, help_text="user that uploads the media"
-    )
+    user = models.ForeignKey("users.User", on_delete=models.CASCADE, help_text="user that uploads the media")
 
     user_featured = models.BooleanField(default=False, help_text="Featured by the user")
 
@@ -406,11 +360,7 @@ class Media(models.Model):
             self.state = helpers.get_default_state(user=self.user)
 
         # condition to appear on listings
-        if (
-            self.state == "public"
-            and self.encoding_status == "success"
-            and self.is_reviewed == True
-        ):
+        if self.state == "public" and self.encoding_status == "success" and self.is_reviewed is True:
             self.listable = True
         else:
             self.listable = False
@@ -419,10 +369,7 @@ class Media(models.Model):
 
         # produce a thumbnail out of an uploaded poster
         # will run only when a poster is uploaded for the first time
-        if (
-            self.uploaded_poster
-            and self.uploaded_poster != self.__original_uploaded_poster
-        ):
+        if self.uploaded_poster and self.uploaded_poster != self.__original_uploaded_poster:
             with open(self.uploaded_poster.path, "rb") as f:
 
                 # set this otherwise gets to infinite loop
@@ -458,9 +405,7 @@ class Media(models.Model):
         ]
         items = [item for item in items if item]
         text = " ".join(items)
-        text = " ".join(
-            [token for token in text.lower().split(" ") if token not in STOP_WORDS]
-        )
+        text = " ".join([token for token in text.lower().split(" ") if token not in STOP_WORDS])
 
         sql_code = """
             UPDATE {db_table} SET search = to_tsvector(
@@ -561,9 +506,7 @@ class Media(models.Model):
             if self.media_type == "image":
                 with open(self.media_file.path, "rb") as f:
                     myfile = File(f)
-                    thumbnail_name = (
-                        helpers.get_file_name(self.media_file.path) + ".jpg"
-                    )
+                    thumbnail_name = helpers.get_file_name(self.media_file.path) + ".jpg"
                     self.thumbnail.save(content=myfile, name=thumbnail_name)
                     self.poster.save(content=myfile, name=thumbnail_name)
         return True
@@ -585,9 +528,7 @@ class Media(models.Model):
         command = [
             settings.FFMPEG_COMMAND,
             "-ss",
-            str(
-                thumbnail_time
-            ),  # -ss need to be firt here otherwise time taken is huge
+            str(thumbnail_time),  # -ss need to be firt here otherwise time taken is huge
             "-i",
             self.media_file.path,
             "-vframes",
@@ -595,7 +536,7 @@ class Media(models.Model):
             "-y",
             tf,
         ]
-        ret = helpers.run_command(command)
+        helpers.run_command(command)
 
         if os.path.exists(tf) and helpers.get_file_type(tf) == "image":
             with open(tf, "rb") as f:
@@ -650,10 +591,7 @@ class Media(models.Model):
             for profile in profiles:
                 if profile.extension != "gif":
                     if self.video_height and self.video_height < profile.resolution:
-                        if (
-                            profile.resolution
-                            not in settings.MINIMUM_RESOLUTIONS_TO_ENCODE
-                        ):
+                        if profile.resolution not in settings.MINIMUM_RESOLUTIONS_TO_ENCODE:
                             continue
                 encoding = Encoding(media=self, profile=profile)
                 encoding.save()
@@ -688,12 +626,7 @@ class Media(models.Model):
 
         self.save(update_fields=["encoding_status", "listable"])
 
-        if (
-            encoding
-            and encoding.status == "success"
-            and encoding.profile.codec == "h264"
-            and action == "add"
-        ):
+        if encoding and encoding.status == "success" and encoding.profile.codec == "h264" and action == "add":
             from . import tasks
 
             tasks.create_hls(self.friendly_token)
@@ -704,10 +637,7 @@ class Media(models.Model):
         """Set encoding_status for videos
         Set success if at least one mp4 exists
         """
-        mp4_statuses = set(
-            encoding.status
-            for encoding in self.encodings.filter(profile__extension="mp4", chunk=False)
-        )
+        mp4_statuses = set(encoding.status for encoding in self.encodings.filter(profile__extension="mp4", chunk=False))
 
         if not mp4_statuses:
             encoding_status = "pending"
@@ -726,7 +656,6 @@ class Media(models.Model):
         """Property used on serializers"""
 
         ret = {}
-        chunks_ret = {}
 
         if self.media_type not in ["video"]:
             return ret
@@ -752,12 +681,8 @@ class Media(models.Model):
                     extra.append(encoding.profile.codec)
             for codec in extra:
                 ret[resolution][codec] = {}
-                v = self.encodings.filter(chunk=True, profile__codec=codec).values(
-                    "progress"
-                )
-                ret[resolution][codec]["progress"] = (
-                    sum([p["progress"] for p in v]) / v.count()
-                )
+                v = self.encodings.filter(chunk=True, profile__codec=codec).values("progress")
+                ret[resolution][codec]["progress"] = sum([p["progress"] for p in v]) / v.count()
                 # TODO; status/logs/errors
         return ret
 
@@ -897,19 +822,13 @@ class Media(models.Model):
                     for iframe_playlist in m3u8_obj.iframe_playlists:
                         uri = os.path.join(p, iframe_playlist.uri)
                         if os.path.exists(uri):
-                            resolution = iframe_playlist.iframe_stream_info.resolution[
-                                1
-                            ]
-                            res["{}_iframe".format(resolution)] = helpers.url_from_path(
-                                uri
-                            )
+                            resolution = iframe_playlist.iframe_stream_info.resolution[1]
+                            res["{}_iframe".format(resolution)] = helpers.url_from_path(uri)
                     for playlist in m3u8_obj.playlists:
                         uri = os.path.join(p, playlist.uri)
                         if os.path.exists(uri):
                             resolution = playlist.stream_info.resolution[1]
-                            res[
-                                "{}_playlist".format(resolution)
-                            ] = helpers.url_from_path(uri)
+                            res["{}_playlist".format(resolution)] = helpers.url_from_path(uri)
         return res
 
     @property
@@ -930,9 +849,7 @@ class Media(models.Model):
         if edit:
             return reverse("edit_media") + "?m={0}".format(self.friendly_token)
         if api:
-            return reverse(
-                "api_get_media", kwargs={"friendly_token": self.friendly_token}
-            )
+            return reverse("api_get_media", kwargs={"friendly_token": self.friendly_token})
         else:
             return reverse("get_media") + "?m={0}".format(self.friendly_token)
 
@@ -988,13 +905,9 @@ class Category(models.Model):
 
     description = models.TextField(blank=True)
 
-    user = models.ForeignKey(
-        "users.User", on_delete=models.CASCADE, blank=True, null=True
-    )
+    user = models.ForeignKey("users.User", on_delete=models.CASCADE, blank=True, null=True)
 
-    is_global = models.BooleanField(
-        default=False, help_text="global categories or user specific"
-    )
+    is_global = models.BooleanField(default=False, help_text="global categories or user specific")
 
     media_count = models.IntegerField(default=0, help_text="number of media")
 
@@ -1006,9 +919,7 @@ class Category(models.Model):
         blank=True,
     )
 
-    listings_thumbnail = models.CharField(
-        max_length=400, blank=True, null=True, help_text="Thumbnail to show on listings"
-    )
+    listings_thumbnail = models.CharField(max_length=400, blank=True, null=True, help_text="Thumbnail to show on listings")
 
     def __str__(self):
         return self.title
@@ -1039,11 +950,7 @@ class Category(models.Model):
         if self.thumbnail:
             return helpers.url_from_path(self.thumbnail.path)
 
-        media = (
-            Media.objects.filter(category=self, state="public")
-            .order_by("-views")
-            .first()
-        )
+        media = Media.objects.filter(category=self, state="public").order_by("-views").first()
         if media:
             return media.thumbnail_url
 
@@ -1061,9 +968,7 @@ class Tag(models.Model):
 
     title = models.CharField(max_length=100, unique=True, db_index=True)
 
-    user = models.ForeignKey(
-        "users.User", on_delete=models.CASCADE, blank=True, null=True
-    )
+    user = models.ForeignKey("users.User", on_delete=models.CASCADE, blank=True, null=True)
 
     media_count = models.IntegerField(default=0, help_text="number of media")
 
@@ -1085,9 +990,7 @@ class Tag(models.Model):
         return reverse("search") + "?t={0}".format(self.title)
 
     def update_tag_media(self):
-        self.media_count = Media.objects.filter(
-            state="public", is_reviewed=True, tags=self
-        ).count()
+        self.media_count = Media.objects.filter(state="public", is_reviewed=True, tags=self).count()
         self.save(update_fields=["media_count"])
         return True
 
@@ -1102,9 +1005,7 @@ class Tag(models.Model):
     def thumbnail_url(self):
         if self.listings_thumbnail:
             return self.listings_thumbnail
-        media = (
-            Media.objects.filter(tags=self, state="public").order_by("-views").first()
-        )
+        media = Media.objects.filter(tags=self, state="public").order_by("-views").first()
         if media:
             return media.thumbnail_url
 
@@ -1154,9 +1055,7 @@ class Encoding(models.Model):
 
     media = models.ForeignKey(Media, on_delete=models.CASCADE, related_name="encodings")
 
-    media_file = models.FileField(
-        "encoding file", upload_to=encoding_media_file_path, blank=True, max_length=500
-    )
+    media_file = models.FileField("encoding file", upload_to=encoding_media_file_path, blank=True, max_length=500)
 
     profile = models.ForeignKey(EncodeProfile, on_delete=models.CASCADE)
 
@@ -1168,9 +1067,7 @@ class Encoding(models.Model):
 
     size = models.CharField(max_length=20, blank=True)
 
-    status = models.CharField(
-        max_length=20, choices=MEDIA_ENCODING_STATUS, default="pending"
-    )
+    status = models.CharField(max_length=20, choices=MEDIA_ENCODING_STATUS, default="pending")
 
     temp_file = models.CharField(max_length=400, blank=True)
 
@@ -1305,9 +1202,7 @@ class Rating(models.Model):
         unique_together = ("user", "media", "rating_category")
 
     def __str__(self):
-        return "{0}, rate for {1} for category {2}".format(
-            self.user.username, self.media.title, self.rating_category.title
-        )
+        return "{0}, rate for {1} for category {2}".format(self.user.username, self.media.title, self.rating_category.title)
 
 
 class Playlist(models.Model):
@@ -1325,9 +1220,7 @@ class Playlist(models.Model):
 
     uid = models.UUIDField(unique=True, default=uuid.uuid4)
 
-    user = models.ForeignKey(
-        "users.User", on_delete=models.CASCADE, db_index=True, related_name="playlists"
-    )
+    user = models.ForeignKey("users.User", on_delete=models.CASCADE, db_index=True, related_name="playlists")
 
     def __str__(self):
         return self.title
@@ -1338,13 +1231,9 @@ class Playlist(models.Model):
 
     def get_absolute_url(self, api=False):
         if api:
-            return reverse(
-                "api_get_playlist", kwargs={"friendly_token": self.friendly_token}
-            )
+            return reverse("api_get_playlist", kwargs={"friendly_token": self.friendly_token})
         else:
-            return reverse(
-                "get_playlist", kwargs={"friendly_token": self.friendly_token}
-            )
+            return reverse("get_playlist", kwargs={"friendly_token": self.friendly_token})
 
     @property
     def url(self):
@@ -1386,7 +1275,7 @@ class Playlist(models.Model):
     @property
     def thumbnail_url(self):
         pm = self.playlistmedia_set.first()
-        if pm:
+        if pm and pm.media.thumbnail:
             return helpers.url_from_path(pm.media.thumbnail.path)
         return None
 
@@ -1411,13 +1300,9 @@ class Comment(MPTTModel):
 
     add_date = models.DateTimeField(auto_now_add=True)
 
-    media = models.ForeignKey(
-        Media, on_delete=models.CASCADE, db_index=True, related_name="comments"
-    )
+    media = models.ForeignKey(Media, on_delete=models.CASCADE, db_index=True, related_name="comments")
 
-    parent = TreeForeignKey(
-        "self", on_delete=models.CASCADE, null=True, blank=True, related_name="children"
-    )
+    parent = TreeForeignKey("self", on_delete=models.CASCADE, null=True, blank=True, related_name="children")
 
     text = models.TextField(help_text="text")
 
@@ -1457,6 +1342,8 @@ def media_save(sender, instance, created, **kwargs):
     # SOS: do not put anything here, as if more logic is added,
     # we have to disconnect signal to avoid infinite recursion
     if created:
+        from .methods import notify_users
+
         instance.media_init()
         notify_users(friendly_token=instance.friendly_token, action="media_added")
 
@@ -1566,13 +1453,9 @@ def encoding_file_save(sender, instance, created, **kwargs):
                 # concatenate chunks and create final encoding file
                 chunks_paths = [f.media_file.path for f in chunks]
 
-                with tempfile.TemporaryDirectory(
-                    dir=settings.TEMP_DIRECTORY
-                ) as temp_dir:
+                with tempfile.TemporaryDirectory(dir=settings.TEMP_DIRECTORY) as temp_dir:
                     seg_file = helpers.create_temp_file(suffix=".txt", dir=temp_dir)
-                    tf = helpers.create_temp_file(
-                        suffix=".{0}".format(instance.profile.extension), dir=temp_dir
-                    )
+                    tf = helpers.create_temp_file(suffix=".{0}".format(instance.profile.extension), dir=temp_dir)
                     with open(seg_file, "w") as ff:
                         for f in chunks_paths:
                             ff.write("file {}\n".format(f))
@@ -1602,9 +1485,7 @@ def encoding_file_save(sender, instance, created, **kwargs):
                         progress=100,
                     )
                     all_logs = "\n".join([st.logs for st in chunks])
-                    encoding.logs = "{0}\n{1}\n{2}".format(
-                        chunks_paths, stdout, all_logs
-                    )
+                    encoding.logs = "{0}\n{1}\n{2}".format(chunks_paths, stdout, all_logs)
                     workers = list(set([st.worker for st in chunks]))
                     encoding.worker = json.dumps({"workers": workers})
 
@@ -1627,7 +1508,7 @@ def encoding_file_save(sender, instance, created, **kwargs):
                     # to avoid that this is run twice
                     if (
                         len(orig_chunks)
-                        == Encoding.objects.filter(
+                        == Encoding.objects.filter(  # noqa
                             media=instance.media,
                             profile=instance.profile,
                             chunks_info=instance.chunks_info,
@@ -1635,9 +1516,7 @@ def encoding_file_save(sender, instance, created, **kwargs):
                     ):
                         # if two chunks are finished at the same time, this
                         # will be changed
-                        who = Encoding.objects.filter(
-                            media=encoding.media, profile=encoding.profile
-                        ).exclude(id=encoding.id)
+                        who = Encoding.objects.filter(media=encoding.media, profile=encoding.profile).exclude(id=encoding.id)
                         who.delete()
                     else:
                         encoding.delete()
@@ -1652,18 +1531,14 @@ def encoding_file_save(sender, instance, created, **kwargs):
                     instance.media.post_encode_actions(encoding=instance, action="add")
 
     elif instance.chunk and instance.status == "fail":
-        encoding = Encoding(
-            media=instance.media, profile=instance.profile, status="fail", progress=100
-        )
+        encoding = Encoding(media=instance.media, profile=instance.profile, status="fail", progress=100)
 
-        chunks = Encoding.objects.filter(
-            media=instance.media, chunks_info=instance.chunks_info, chunk=True
-        ).order_by("add_date")
+        chunks = Encoding.objects.filter(media=instance.media, chunks_info=instance.chunks_info, chunk=True).order_by("add_date")
 
         chunks_paths = [f.media_file.path for f in chunks]
 
         all_logs = "\n".join([st.logs for st in chunks])
-        encoding.logs = "{0}\n{1}\n{2}".format(chunks_paths, all_logs)
+        encoding.logs = "{0}\n{1}".format(chunks_paths, all_logs)
         workers = list(set([st.worker for st in chunks]))
         encoding.worker = json.dumps({"workers": workers})
         start_date = min([st.add_date for st in chunks])
@@ -1671,32 +1546,17 @@ def encoding_file_save(sender, instance, created, **kwargs):
         encoding.total_run_time = (end_date - start_date).seconds
         encoding.save()
 
-        who = Encoding.objects.filter(
-            media=encoding.media, profile=encoding.profile
-        ).exclude(id=encoding.id)
+        who = Encoding.objects.filter(media=encoding.media, profile=encoding.profile).exclude(id=encoding.id)
 
         who.delete()
-        pass  # TODO: merge with above if, do not repeat code
+        # TODO: merge with above if, do not repeat code
     else:
         if instance.status in ["fail", "success"]:
             instance.media.post_encode_actions(encoding=instance, action="add")
 
-        encodings = set(
-            [
-                encoding.status
-                for encoding in Encoding.objects.filter(media=instance.media)
-            ]
-        )
+        encodings = set([encoding.status for encoding in Encoding.objects.filter(media=instance.media)])
         if ("running" in encodings) or ("pending" in encodings):
             return
-        workers = list(
-            set(
-                [
-                    encoding.worker
-                    for encoding in Encoding.objects.filter(media=instance.media)
-                ]
-            )
-        )
 
 
 @receiver(post_delete, sender=Encoding)
@@ -1712,4 +1572,3 @@ def encoding_file_delete(sender, instance, **kwargs):
             instance.media.post_encode_actions(encoding=instance, action="delete")
     # delete local chunks, and remote chunks + media file. Only when the
     # last encoding of a media is complete
-

files/permissions.py

@@ -1,4 +1,5 @@
 from rest_framework import permissions
+
 from .methods import is_mediacms_editor
 
 

files/serializers.py

@@ -1,6 +1,6 @@
 from rest_framework import serializers
 
-from .models import Media, EncodeProfile, Playlist, Comment, Category, Tag
+from .models import Category, Comment, EncodeProfile, Media, Playlist, Tag
 
 # TODO: put them in a more DRY way
 
@@ -18,9 +18,7 @@ class MediaSerializer(serializers.ModelSerializer):
         return self.context["request"].build_absolute_uri(obj.get_absolute_url())
 
     def get_api_url(self, obj):
-        return self.context["request"].build_absolute_uri(
-            obj.get_absolute_url(api=True)
-        )
+        return self.context["request"].build_absolute_uri(obj.get_absolute_url(api=True))
 
     def get_thumbnail_url(self, obj):
         if obj.thumbnail_url:
@@ -40,7 +38,6 @@ class MediaSerializer(serializers.ModelSerializer):
             "friendly_token",
             "user",
             "add_date",
-            "views",
             "media_type",
             "state",
             "duration",
@@ -51,6 +48,7 @@ class MediaSerializer(serializers.ModelSerializer):
             "reported_times",
             "size",
             "is_reviewed",
+            "featured",
         )
         fields = (
             "friendly_token",
@@ -66,8 +64,6 @@ class MediaSerializer(serializers.ModelSerializer):
             "duration",
             "thumbnail_url",
             "is_reviewed",
-            "url",
-            "api_url",
             "preview_url",
             "author_name",
             "author_profile",
@@ -210,16 +206,7 @@ class PlaylistSerializer(serializers.ModelSerializer):
     class Meta:
         model = Playlist
         read_only_fields = ("add_date", "user")
-        fields = (
-            "add_date",
-            "title",
-            "description",
-            "user",
-            "media_count",
-            "url",
-            "api_url",
-            "thumbnail_url"
-        )
+        fields = ("add_date", "title", "description", "user", "media_count", "url", "api_url", "thumbnail_url")
 
 
 class PlaylistDetailSerializer(serializers.ModelSerializer):
@@ -228,16 +215,7 @@ class PlaylistDetailSerializer(serializers.ModelSerializer):
     class Meta:
         model = Playlist
         read_only_fields = ("add_date", "user")
-        fields = (
-            "title",
-            "add_date",
-            "user_thumbnail_url",
-            "description",
-            "user",
-            "media_count",
-            "url",
-            "thumbnail_url"
-        )
+        fields = ("title", "add_date", "user_thumbnail_url", "description", "user", "media_count", "url", "thumbnail_url")
 
 
 class CommentSerializer(serializers.ModelSerializer):

files/tasks.py

@@ -1,41 +1,40 @@
-import re
-import os
 import json
-import subprocess
-from datetime import datetime, timedelta
-import tempfile
+import os
+import re
 import shutil
-from django.core.cache import cache
-from django.conf import settings
-from django.core.files import File
-from django.db.models import Q
+import subprocess
+import tempfile
+from datetime import datetime, timedelta
 
 from celery import Task
 from celery.decorators import task
-from celery.utils.log import get_task_logger
 from celery.exceptions import SoftTimeLimitExceeded
-
-from celery.task.control import revoke
 from celery.signals import task_revoked
+from celery.task.control import revoke
+from celery.utils.log import get_task_logger
+from django.conf import settings
+from django.core.cache import cache
+from django.core.files import File
+from django.db.models import Q
+
+from actions.models import USER_MEDIA_ACTIONS, MediaAction
+from users.models import User
 
 from .backends import FFmpegBackend
 from .exceptions import VideoEncodingError
 from .helpers import (
     calculate_seconds,
-    rm_file,
     create_temp_file,
     get_file_name,
     get_file_type,
     media_file_info,
-    run_command,
     produce_ffmpeg_commands,
     produce_friendly_token,
+    rm_file,
+    run_command,
 )
-
-from actions.models import MediaAction, USER_MEDIA_ACTIONS
-from users.models import User
-from .models import Encoding, EncodeProfile, Media, Category, Rating, Tag
-from .methods import list_tasks, pre_save_action, notify_users
+from .methods import list_tasks, notify_users, pre_save_action
+from .models import Category, EncodeProfile, Encoding, Media, Rating, Tag
 
 logger = get_task_logger(__name__)
 
@@ -83,10 +82,7 @@ def chunkize_media(self, friendly_token, profiles, force=True):
                 chunks.append(ch[0])
     if not chunks:
         # command completely failed to segment file.putting to normal encode
-        logger.info(
-            "Failed to break file {0} in chunks."
-            " Putting to normal encode queue".format(friendly_token)
-        )
+        logger.info("Failed to break file {0} in chunks." " Putting to normal encode queue".format(friendly_token))
         for profile in profiles:
             if media.video_height and media.video_height < profile.resolution:
                 if profile.resolution not in settings.MINIMUM_RESOLUTIONS_TO_ENCODE:
@@ -94,9 +90,7 @@ def chunkize_media(self, friendly_token, profiles, force=True):
             encoding = Encoding(media=media, profile=profile)
             encoding.save()
             enc_url = settings.SSL_FRONTEND_HOST + encoding.get_absolute_url()
-            encode_media.delay(
-                friendly_token, profile.id, encoding.id, enc_url, force=force
-            )
+            encode_media.delay(friendly_token, profile.id, encoding.id, enc_url, force=force)
         return False
 
     chunks = [os.path.join(cwd, ch) for ch in chunks]
@@ -137,11 +131,7 @@ def chunkize_media(self, friendly_token, profiles, force=True):
                 priority=priority,
             )
 
-    logger.info(
-        "got {0} chunks and will encode to {1} profiles".format(
-            len(chunks), to_profiles
-        )
-    )
+    logger.info("got {0} chunks and will encode to {1} profiles".format(len(chunks), to_profiles))
     return True
 
 
@@ -180,11 +170,7 @@ def encode_media(
 ):
     """Encode a media to given profile, using ffmpeg, storing progress"""
 
-    logger.info(
-        "Encode Media started, friendly token {0}, profile id {1}, force {2}".format(
-            friendly_token, profile_id, force
-        )
-    )
+    logger.info("Encode Media started, friendly token {0}, profile id {1}, force {2}".format(friendly_token, profile_id, force))
 
     if self.request.id:
         task_id = self.request.id
@@ -202,13 +188,7 @@ def encode_media(
         # TODO: in case a video is chunkized and this enters here many times
         # it will always run since chunk_file_path is always different
         # thus find a better way for this check
-        if (
-            Encoding.objects.filter(
-                media=media, profile=profile, chunk_file_path=chunk_file_path
-            ).count()
-            > 1
-            and force == False
-        ):
+        if Encoding.objects.filter(media=media, profile=profile, chunk_file_path=chunk_file_path).count() > 1 and force is False:
             Encoding.objects.filter(id=encoding_id).delete()
             return False
         else:
@@ -230,19 +210,14 @@ def encode_media(
                     chunk_file_path=chunk_file_path,
                 )
     else:
-        if (
-            Encoding.objects.filter(media=media, profile=profile).count() > 1
-            and force is False
-        ):
+        if Encoding.objects.filter(media=media, profile=profile).count() > 1 and force is False:
             Encoding.objects.filter(id=encoding_id).delete()
             return False
         else:
             try:
                 encoding = Encoding.objects.get(id=encoding_id)
                 encoding.status = "running"
-                Encoding.objects.filter(media=media, profile=profile).exclude(
-                    id=encoding_id
-                ).delete()
+                Encoding.objects.filter(media=media, profile=profile).exclude(id=encoding_id).delete()
             except BaseException:
                 encoding = Encoding(media=media, profile=profile, status="running")
 
@@ -287,10 +262,10 @@ def encode_media(
     else:
         original_media_path = media.media_file.path
 
-    if not media.duration:
-        encoding.status = "fail"
-        encoding.save(update_fields=["status"])
-        return False
+    # if not media.duration:
+    #    encoding.status = "fail"
+    #    encoding.save(update_fields=["status"])
+    #    return False
 
     with tempfile.TemporaryDirectory(dir=settings.TEMP_DIRECTORY) as temp_dir:
 
@@ -337,9 +312,7 @@ def encode_media(
                             if n_times % 60 == 0:
                                 encoding.progress = percent
                                 try:
-                                    encoding.save(
-                                        update_fields=["progress", "update_date"]
-                                    )
+                                    encoding.save(update_fields=["progress", "update_date"])
                                     logger.info("Saved {0}".format(round(percent, 2)))
                                 except BaseException:
                                     pass
@@ -383,18 +356,12 @@ def encode_media(
 
                 with open(tf, "rb") as f:
                     myfile = File(f)
-                    output_name = "{0}.{1}".format(
-                        get_file_name(original_media_path), profile.extension
-                    )
+                    output_name = "{0}.{1}".format(get_file_name(original_media_path), profile.extension)
                     encoding.media_file.save(content=myfile, name=output_name)
-                encoding.total_run_time = (
-                    encoding.update_date - encoding.add_date
-                ).seconds
+                encoding.total_run_time = (encoding.update_date - encoding.add_date).seconds
 
         try:
-            encoding.save(
-                update_fields=["status", "logs", "progress", "total_run_time"]
-            )
+            encoding.save(update_fields=["status", "logs", "progress", "total_run_time"])
         # this will raise a django.db.utils.DatabaseError error when task is revoked,
         # since we delete the encoding at that stage
         except BaseException:
@@ -424,7 +391,7 @@ def produce_sprite_from_video(friendly_token):
                 tmpdirname,
                 output_name,
             )
-            ret = subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
+            subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
             if os.path.exists(output_name) and get_file_type(output_name) == "image":
                 with open(output_name, "rb") as f:
                     myfile = File(f)
@@ -457,23 +424,19 @@ def create_hls(friendly_token):
 
     p = media.uid.hex
     output_dir = os.path.join(settings.HLS_DIR, p)
-    encodings = media.encodings.filter(
-        profile__extension="mp4", status="success", chunk=False, profile__codec="h264"
-    )
+    encodings = media.encodings.filter(profile__extension="mp4", status="success", chunk=False, profile__codec="h264")
     if encodings:
         existing_output_dir = None
         if os.path.exists(output_dir):
             existing_output_dir = output_dir
             output_dir = os.path.join(settings.HLS_DIR, p + produce_friendly_token())
         files = " ".join([f.media_file.path for f in encodings if f.media_file])
-        cmd = "{0} --segment-duration=4 --output-dir={1} {2}".format(
-            settings.MP4HLS_COMMAND, output_dir, files
-        )
-        ret = subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
+        cmd = "{0} --segment-duration=4 --output-dir={1} {2}".format(settings.MP4HLS_COMMAND, output_dir, files)
+        subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
         if existing_output_dir:
             # override content with -T !
             cmd = "cp -rT {0} {1}".format(output_dir, existing_output_dir)
-            ret = subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
+            subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
             shutil.rmtree(output_dir)
             output_dir = existing_output_dir
         pp = os.path.join(output_dir, "master.m3u8")
@@ -515,11 +478,7 @@ def check_running_states():
 def check_media_states():
     # Experimental - unused
     # check encoding status of not success media
-    media = Media.objects.filter(
-        Q(encoding_status="running")
-        | Q(encoding_status="fail")
-        | Q(encoding_status="pending")
-    )
+    media = Media.objects.filter(Q(encoding_status="running") | Q(encoding_status="fail") | Q(encoding_status="pending"))
 
     logger.info("got {0} media that are not in state success".format(media.count()))
 
@@ -564,11 +523,7 @@ def check_pending_states():
             media.encode(profiles=[profile], force=False)
             changed += 1
     if changed:
-        logger.info(
-            "set to the encode queue {0} encodings that were on pending state".format(
-                changed
-            )
-        )
+        logger.info("set to the encode queue {0} encodings that were on pending state".format(changed))
     return True
 
 
@@ -602,6 +557,7 @@ def clear_sessions():
 
     try:
         from importlib import import_module
+
         from django.conf import settings
 
         engine = import_module(settings.SESSION_ENGINE)
@@ -612,9 +568,7 @@ def clear_sessions():
 
 
 @task(name="save_user_action", queue="short_tasks")
-def save_user_action(
-    user_or_session, friendly_token=None, action="watch", extra_info=None
-):
+def save_user_action(user_or_session, friendly_token=None, action="watch", extra_info=None):
     """Short task that saves a user action"""
 
     if action not in VALID_USER_ACTIONS:
@@ -638,7 +592,7 @@ def save_user_action(
     if not (user or session_key):
         return False
 
-    if action in ["like", "dislike", "report"]:
+    if action in ["like", "dislike", "watch", "report"]:
         if not pre_save_action(
             media=media,
             user=user,
@@ -652,9 +606,7 @@ def save_user_action(
         if user:
             MediaAction.objects.filter(user=user, media=media, action="watch").delete()
         else:
-            MediaAction.objects.filter(
-                session_key=session_key, media=media, action="watch"
-            ).delete()
+            MediaAction.objects.filter(session_key=session_key, media=media, action="watch").delete()
     if action == "rate":
         try:
             score = extra_info.get("score")
@@ -663,9 +615,7 @@ def save_user_action(
             # TODO: better error handling?
             return False
         try:
-            rating = Rating.objects.filter(
-                user=user, media=media, rating_category_id=rating_category
-            ).first()
+            rating = Rating.objects.filter(user=user, media=media, rating_category_id=rating_category).first()
             if rating:
                 rating.score = score
                 rating.save(update_fields=["score"])
@@ -676,7 +626,7 @@ def save_user_action(
                     rating_category_id=rating_category,
                     score=score,
                 )
-        except Exception as exc:
+        except Exception:
             # TODO: more specific handling, for errors in score, or
             # rating_category?
             return False
@@ -735,14 +685,10 @@ def get_list_of_popular_media():
 
     for media in media_x:
         ft = media["friendly_token"]
-        num = MediaAction.objects.filter(
-            action_date__gte=period_x, action="watch", media__friendly_token=ft
-        ).count()
+        num = MediaAction.objects.filter(action_date__gte=period_x, action="watch", media__friendly_token=ft).count()
         if num:
             valid_media_x[ft] = num
-        num = MediaAction.objects.filter(
-            action_date__gte=period_y, action="like", media__friendly_token=ft
-        ).count()
+        num = MediaAction.objects.filter(action_date__gte=period_y, action="like", media__friendly_token=ft).count()
         if num:
             valid_media_y[ft] = num
 
@@ -767,12 +713,7 @@ def update_listings_thumbnails():
     saved = 0
     qs = Category.objects.filter().order_by("-media_count")
     for object in qs:
-        media = (
-            Media.objects.exclude(friendly_token__in=used_media)
-            .filter(category=object, state="public", is_reviewed=True)
-            .order_by("-views")
-            .first()
-        )
+        media = Media.objects.exclude(friendly_token__in=used_media).filter(category=object, state="public", is_reviewed=True).order_by("-views").first()
         if media:
             object.listings_thumbnail = media.thumbnail_url
             object.save(update_fields=["listings_thumbnail"])
@@ -785,12 +726,7 @@ def update_listings_thumbnails():
     saved = 0
     qs = Tag.objects.filter().order_by("-media_count")
     for object in qs:
-        media = (
-            Media.objects.exclude(friendly_token__in=used_media)
-            .filter(tags=object, state="public", is_reviewed=True)
-            .order_by("-views")
-            .first()
-        )
+        media = Media.objects.exclude(friendly_token__in=used_media).filter(tags=object, state="public", is_reviewed=True).order_by("-views").first()
         if media:
             object.listings_thumbnail = media.thumbnail_url
             object.save(update_fields=["listings_thumbnail"])

files/urls.py

@@ -1,10 +1,9 @@
-from django.conf.urls.static import static
 from django.conf import settings
-from django.conf.urls import url, include
+from django.conf.urls import include, url
+from django.conf.urls.static import static
 from django.urls import path
 
-from . import views
-from . import management_views
+from . import management_views, views
 from .feeds import IndexRSSFeed, SearchRSSFeed
 
 urlpatterns = [

files/views.py

@@ -1,69 +1,69 @@
 from datetime import datetime, timedelta
-from django.shortcuts import render
-from django.http import HttpResponseRedirect
-from django.conf import settings
-from django.shortcuts import get_object_or_404
-from django.db.models import Q
-from django.contrib.auth.decorators import login_required
-from django.contrib import messages
-from django.template.defaultfilters import slugify
-from django.core.mail import EmailMessage
-from django.contrib.postgres.search import SearchQuery
-
-from rest_framework import permissions
-from rest_framework.views import APIView
-from rest_framework.response import Response
-from rest_framework.settings import api_settings
-from rest_framework.exceptions import PermissionDenied
-from rest_framework import status
-from rest_framework.parsers import (
-    JSONParser,
-    MultiPartParser,
-    FileUploadParser,
-    FormParser,
-)
 
 from celery.task.control import revoke
-from cms.permissions import IsAuthorizedToAdd, IsUserOrEditor
-from cms.permissions import user_allowed_to_upload
-from cms.custom_pagination import FastPaginationWithoutCount
-from actions.models import MediaAction, USER_MEDIA_ACTIONS
-from users.models import User
-from .helpers import produce_ffmpeg_commands, clean_query
-from .models import (
-    Media,
-    EncodeProfile,
-    Encoding,
-    Playlist,
-    PlaylistMedia,
-    Comment,
-    Category,
-    Tag,
+from django.conf import settings
+from django.contrib import messages
+from django.contrib.auth.decorators import login_required
+from django.contrib.postgres.search import SearchQuery
+from django.core.mail import EmailMessage
+from django.db.models import Q
+from django.http import HttpResponseRedirect
+from django.shortcuts import get_object_or_404, render
+from django.template.defaultfilters import slugify
+from drf_yasg import openapi as openapi
+from drf_yasg.utils import swagger_auto_schema
+from rest_framework import permissions, status
+from rest_framework.exceptions import PermissionDenied
+from rest_framework.parsers import (
+    FileUploadParser,
+    FormParser,
+    JSONParser,
+    MultiPartParser,
 )
-from .forms import MediaForm, ContactForm, SubtitleForm
-from .tasks import save_user_action
+from rest_framework.response import Response
+from rest_framework.settings import api_settings
+from rest_framework.views import APIView
+
+from actions.models import USER_MEDIA_ACTIONS, MediaAction
+from cms.custom_pagination import FastPaginationWithoutCount
+from cms.permissions import IsAuthorizedToAdd, IsUserOrEditor, user_allowed_to_upload
+from users.models import User
+
+from .forms import ContactForm, MediaForm, SubtitleForm
+from .helpers import clean_query, produce_ffmpeg_commands
 from .methods import (
-    list_tasks,
     get_user_or_session,
-    show_recommended_media,
-    show_related_media,
     is_mediacms_editor,
     is_mediacms_manager,
-    update_user_ratings,
+    list_tasks,
     notify_user_on_comment,
+    show_recommended_media,
+    show_related_media,
+    update_user_ratings,
+)
+from .models import (
+    Category,
+    Comment,
+    EncodeProfile,
+    Encoding,
+    Media,
+    Playlist,
+    PlaylistMedia,
+    Tag,
 )
 from .serializers import (
-    MediaSerializer,
     CategorySerializer,
-    TagSerializer,
-    SingleMediaSerializer,
+    CommentSerializer,
     EncodeProfileSerializer,
     MediaSearchSerializer,
-    PlaylistSerializer,
+    MediaSerializer,
     PlaylistDetailSerializer,
-    CommentSerializer,
+    PlaylistSerializer,
+    SingleMediaSerializer,
+    TagSerializer,
 )
 from .stop_words import STOP_WORDS
+from .tasks import save_user_action
 
 VALID_USER_ACTIONS = [action for action, name in USER_MEDIA_ACTIONS]
 
@@ -86,11 +86,7 @@ def add_subtitle(request):
     if not media:
         return HttpResponseRedirect("/")
 
-    if not (
-        request.user == media.user
-        or is_mediacms_editor(request.user)
-        or is_mediacms_manager(request.user)
-    ):
+    if not (request.user == media.user or is_mediacms_editor(request.user) or is_mediacms_manager(request.user)):
         return HttpResponseRedirect("/")
 
     if request.method == "POST":
@@ -175,11 +171,7 @@ def edit_media(request):
     if not media:
         return HttpResponseRedirect("/")
 
-    if not (
-        request.user == media.user
-        or is_mediacms_editor(request.user)
-        or is_mediacms_manager(request.user)
-    ):
+    if not (request.user == media.user or is_mediacms_editor(request.user) or is_mediacms_manager(request.user)):
         return HttpResponseRedirect("/")
     if request.method == "POST":
         form = MediaForm(request.user, request.POST, request.FILES, instance=media)
@@ -220,8 +212,6 @@ def embed_media(request):
     if not media:
         return HttpResponseRedirect("/")
 
-    user_or_session = get_user_or_session(request)
-
     context = {}
     context["media"] = friendly_token
     return render(request, "cms/embed.html", context)
@@ -342,9 +332,7 @@ def view_media(request):
         return render(request, "cms/media.html", context)
 
     user_or_session = get_user_or_session(request)
-    save_user_action.delay(
-        user_or_session, friendly_token=friendly_token, action="watch"
-    )
+    save_user_action.delay(user_or_session, friendly_token=friendly_token, action="watch")
     context = {}
     context["media"] = friendly_token
     context["media_object"] = media
@@ -354,11 +342,7 @@ def view_media(request):
     context["CAN_DELETE_COMMENTS"] = False
 
     if request.user.is_authenticated:
-        if (
-            (media.user.id == request.user.id)
-            or is_mediacms_editor(request.user)
-            or is_mediacms_manager(request.user)
-        ):
+        if (media.user.id == request.user.id) or is_mediacms_editor(request.user) or is_mediacms_manager(request.user):
             context["CAN_DELETE_MEDIA"] = True
             context["CAN_EDIT_MEDIA"] = True
             context["CAN_DELETE_COMMENTS"] = True
@@ -382,8 +366,19 @@ class MediaList(APIView):
     """Media listings views"""
 
     permission_classes = (IsAuthorizedToAdd,)
-    parser_classes = (JSONParser, MultiPartParser, FormParser, FileUploadParser)
+    parser_classes = (MultiPartParser, FormParser, FileUploadParser)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='page', type=openapi.TYPE_INTEGER, in_=openapi.IN_QUERY, description='Page number'),
+            openapi.Parameter(name='author', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='username'),
+            openapi.Parameter(name='show', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='show', enum=['recommended', 'featured', 'latest']),
+        ],
+        tags=['Media'],
+        operation_summary='List Media',
+        operation_description='Lists all media',
+        responses={200: MediaSerializer(many=True)},
+    )
     def get(self, request, format=None):
         # Show media
         params = self.request.query_params
@@ -423,6 +418,17 @@ class MediaList(APIView):
         serializer = MediaSerializer(page, many=True, context={"request": request})
         return paginator.get_paginated_response(serializer.data)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name="media_file", in_=openapi.IN_FORM, type=openapi.TYPE_FILE, required=True, description="media_file"),
+            openapi.Parameter(name="description", in_=openapi.IN_FORM, type=openapi.TYPE_STRING, required=False, description="description"),
+            openapi.Parameter(name="title", in_=openapi.IN_FORM, type=openapi.TYPE_STRING, required=False, description="title"),
+        ],
+        tags=['Media'],
+        operation_summary='Add new Media',
+        operation_description='Adds a new media, for authenticated users',
+        responses={201: openapi.Response('response description', MediaSerializer), 401: 'bad request'},
+    )
     def post(self, request, format=None):
         # Add new media
         serializer = MediaSerializer(data=request.data, context={"request": request})
@@ -439,43 +445,40 @@ class MediaDetail(APIView):
     """
 
     permission_classes = (permissions.IsAuthenticatedOrReadOnly, IsUserOrEditor)
-    parser_classes = (JSONParser, MultiPartParser, FormParser, FileUploadParser)
+    parser_classes = (MultiPartParser, FormParser, FileUploadParser)
 
     def get_object(self, friendly_token, password=None):
         try:
-            media = (
-                Media.objects.select_related("user")
-                .prefetch_related("encodings__profile")
-                .get(friendly_token=friendly_token)
-            )
+            media = Media.objects.select_related("user").prefetch_related("encodings__profile").get(friendly_token=friendly_token)
 
             # this need be explicitly called, and will call
             # has_object_permission() after has_permission has succeeded
             self.check_object_permissions(self.request, media)
 
-            if media.state == "private" and not (
-                self.request.user == media.user or is_mediacms_editor(self.request.user)
-            ):
-                if (
-                    (not password)
-                    or (not media.password)
-                    or (password != media.password)
-                ):
+            if media.state == "private" and not (self.request.user == media.user or is_mediacms_editor(self.request.user)):
+                if (not password) or (not media.password) or (password != media.password):
                     return Response(
                         {"detail": "media is private"},
                         status=status.HTTP_401_UNAUTHORIZED,
                     )
             return media
         except PermissionDenied:
-            return Response(
-                {"detail": "bad permissions"}, status=status.HTTP_401_UNAUTHORIZED
-            )
+            return Response({"detail": "bad permissions"}, status=status.HTTP_401_UNAUTHORIZED)
         except BaseException:
             return Response(
                 {"detail": "media file does not exist"},
                 status=status.HTTP_400_BAD_REQUEST,
             )
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='friendly_token', type=openapi.TYPE_STRING, in_=openapi.IN_PATH, description='unique identifier', required=True),
+        ],
+        tags=['Media'],
+        operation_summary='Get information for Media',
+        operation_description='Get information for a media',
+        responses={200: SingleMediaSerializer(), 400: 'bad request'},
+    )
     def get(self, request, friendly_token, format=None):
         # Get media details
         password = request.GET.get("password")
@@ -488,27 +491,38 @@ class MediaDetail(APIView):
             related_media = []
         else:
             related_media = show_related_media(media, request=request, limit=100)
-            related_media_serializer = MediaSerializer(
-                related_media, many=True, context={"request": request}
-            )
+            related_media_serializer = MediaSerializer(related_media, many=True, context={"request": request})
             related_media = related_media_serializer.data
         ret = serializer.data
 
         # update rattings info with user specific ratings
         # eg user has already rated for this media
         # this only affects user rating and only if enabled
-        if (
-            settings.ALLOW_RATINGS
-            and ret.get("ratings_info")
-            and not request.user.is_anonymous
-        ):
-            ret["ratings_info"] = update_user_ratings(
-                request.user, media, ret.get("ratings_info")
-            )
+        if settings.ALLOW_RATINGS and ret.get("ratings_info") and not request.user.is_anonymous:
+            ret["ratings_info"] = update_user_ratings(request.user, media, ret.get("ratings_info"))
 
         ret["related_media"] = related_media
         return Response(ret)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='friendly_token', type=openapi.TYPE_STRING, in_=openapi.IN_PATH, description='unique identifier', required=True),
+            openapi.Parameter(name='type', type=openapi.TYPE_STRING, in_=openapi.IN_FORM, description='action to perform', enum=['encode', 'review']),
+            openapi.Parameter(
+                name='encoding_profiles',
+                type=openapi.TYPE_ARRAY,
+                items=openapi.Items(type=openapi.TYPE_STRING),
+                in_=openapi.IN_FORM,
+                description='if action to perform is encode, need to specify list of ids of encoding profiles',
+            ),
+            openapi.Parameter(name='result', type=openapi.TYPE_BOOLEAN, in_=openapi.IN_FORM, description='if action is review, this is the result (True for reviewed, False for not reviewed)'),
+        ],
+        tags=['Media'],
+        operation_summary='Run action on Media',
+        operation_description='Actions for a media, for MediaCMS editors and managers',
+        responses={201: 'action created', 400: 'bad request'},
+        operation_id='media_manager_actions',
+    )
     def post(self, request, friendly_token, format=None):
         """superuser actions
         Available only to MediaCMS editors and managers
@@ -521,14 +535,11 @@ class MediaDetail(APIView):
             return media
 
         if not (is_mediacms_editor(request.user) or is_mediacms_manager(request.user)):
-            return Response(
-                {"detail": "not allowed"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "not allowed"}, status=status.HTTP_400_BAD_REQUEST)
 
         action = request.data.get("type")
         profiles_list = request.data.get("encoding_profiles")
         result = request.data.get("result", True)
-
         if action == "encode":
             # Create encoding tasks for specific profiles
             valid_profiles = []
@@ -544,44 +555,61 @@ class MediaDetail(APIView):
                         valid_profiles.append(p)
                     except ValueError:
                         return Response(
-                            {
-                                "detail": "encoding_profiles must be int or list of ints of valid encode profiles"
-                            },
+                            {"detail": "encoding_profiles must be int or list of ints of valid encode profiles"},
                             status=status.HTTP_400_BAD_REQUEST,
                         )
             media.encode(profiles=valid_profiles)
-            return Response(
-                {"detail": "media will be encoded"}, status=status.HTTP_201_CREATED
-            )
+            return Response({"detail": "media will be encoded"}, status=status.HTTP_201_CREATED)
         elif action == "review":
             if result:
                 media.is_reviewed = True
-            elif result == False:
+            elif result is False:
                 media.is_reviewed = False
             media.save(update_fields=["is_reviewed"])
-            return Response(
-                {"detail": "media reviewed set"}, status=status.HTTP_201_CREATED
-            )
+            return Response({"detail": "media reviewed set"}, status=status.HTTP_201_CREATED)
         return Response(
             {"detail": "not valid action or no action specified"},
             status=status.HTTP_400_BAD_REQUEST,
         )
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name="description", in_=openapi.IN_FORM, type=openapi.TYPE_STRING, required=False, description="description"),
+            openapi.Parameter(name="title", in_=openapi.IN_FORM, type=openapi.TYPE_STRING, required=False, description="title"),
+            openapi.Parameter(name="media_file", in_=openapi.IN_FORM, type=openapi.TYPE_FILE, required=False, description="media_file"),
+        ],
+        tags=['Media'],
+        operation_summary='Update Media',
+        operation_description='Update a Media, for Media uploader',
+        responses={201: openapi.Response('response description', MediaSerializer), 401: 'bad request'},
+    )
     def put(self, request, friendly_token, format=None):
         # Update a media object
         media = self.get_object(friendly_token)
         if isinstance(media, Response):
             return media
 
-        serializer = MediaSerializer(
-            media, data=request.data, context={"request": request}
-        )
+        serializer = MediaSerializer(media, data=request.data, context={"request": request})
         if serializer.is_valid():
-            media_file = request.data["media_file"]
-            serializer.save(user=request.user, media_file=media_file)
+            if request.data.get('media_file'):
+                media_file = request.data["media_file"]
+                serializer.save(user=request.user, media_file=media_file)
+            else:
+                serializer.save(user=request.user)
             return Response(serializer.data, status=status.HTTP_201_CREATED)
         return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='friendly_token', type=openapi.TYPE_STRING, in_=openapi.IN_PATH, description='unique identifier', required=True),
+        ],
+        tags=['Media'],
+        operation_summary='Delete Media',
+        operation_description='Delete a Media, for MediaCMS editors and managers',
+        responses={
+            204: 'no content',
+        },
+    )
     def delete(self, request, friendly_token, format=None):
         # Delete a media object
         media = self.get_object(friendly_token)
@@ -601,26 +629,24 @@ class MediaActions(APIView):
 
     def get_object(self, friendly_token):
         try:
-            media = (
-                Media.objects.select_related("user")
-                .prefetch_related("encodings__profile")
-                .get(friendly_token=friendly_token)
-            )
+            media = Media.objects.select_related("user").prefetch_related("encodings__profile").get(friendly_token=friendly_token)
             if media.state == "private" and self.request.user != media.user:
-                return Response(
-                    {"detail": "media is private"}, status=status.HTTP_400_BAD_REQUEST
-                )
+                return Response({"detail": "media is private"}, status=status.HTTP_400_BAD_REQUEST)
             return media
         except PermissionDenied:
-            return Response(
-                {"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST)
         except BaseException:
             return Response(
                 {"detail": "media file does not exist"},
                 status=status.HTTP_400_BAD_REQUEST,
             )
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Media'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def get(self, request, friendly_token, format=None):
         # show date and reason for each time media was reported
         media = self.get_object(friendly_token)
@@ -636,6 +662,12 @@ class MediaActions(APIView):
 
         return Response(ret, status=status.HTTP_200_OK)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Media'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def post(self, request, friendly_token, format=None):
         # perform like/dislike/report actions
         media = self.get_object(friendly_token)
@@ -661,23 +693,23 @@ class MediaActions(APIView):
                 extra_info=extra,
             )
 
-            return Response(
-                {"detail": "action received"}, status=status.HTTP_201_CREATED
-            )
+            return Response({"detail": "action received"}, status=status.HTTP_201_CREATED)
         else:
-            return Response(
-                {"detail": "no action specified"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "no action specified"}, status=status.HTTP_400_BAD_REQUEST)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Media'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def delete(self, request, friendly_token, format=None):
         media = self.get_object(friendly_token)
         if isinstance(media, Response):
             return media
 
         if not request.user.is_superuser:
-            return Response(
-                {"detail": "not allowed"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "not allowed"}, status=status.HTTP_400_BAD_REQUEST)
 
         action = request.data.get("type")
         if action:
@@ -690,9 +722,7 @@ class MediaActions(APIView):
                     status=status.HTTP_201_CREATED,
                 )
         else:
-            return Response(
-                {"detail": "no action specified"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "no action specified"}, status=status.HTTP_400_BAD_REQUEST)
 
 
 class MediaSearch(APIView):
@@ -703,6 +733,12 @@ class MediaSearch(APIView):
 
     parser_classes = (JSONParser,)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Search'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def get(self, request, format=None):
         params = self.request.query_params
         query = params.get("q", "").strip().lower()
@@ -736,11 +772,7 @@ class MediaSearch(APIView):
         if query:
             # move this processing to a prepare_query function
             query = clean_query(query)
-            q_parts = [
-                q_part.rstrip("y")
-                for q_part in query.split()
-                if q_part not in STOP_WORDS
-            ]
+            q_parts = [q_part.rstrip("y") for q_part in query.split() if q_part not in STOP_WORDS]
             if q_parts:
                 query = SearchQuery(q_parts[0] + ":*", search_type="raw")
                 for part in q_parts[1:]:
@@ -763,7 +795,7 @@ class MediaSearch(APIView):
             media = media.filter(user__username=author)
 
         if upload_date:
-            gte = lte = None
+            gte = None
             if upload_date == 'today':
                 gte = datetime.now().date()
             if upload_date == 'this_week':
@@ -771,12 +803,10 @@ class MediaSearch(APIView):
             if upload_date == 'this_month':
                 year = datetime.now().date().year
                 month = datetime.now().date().month
-                gte = datetime(year,month,1)
+                gte = datetime(year, month, 1)
             if upload_date == 'this_year':
                 year = datetime.now().date().year
-                gte = datetime(year,1,1)
-            if lte:
-                media = media.filter(add_date__lte=lte)
+                gte = datetime(year, 1, 1)
             if gte:
                 media = media.filter(add_date__gte=gte)
 
@@ -794,9 +824,7 @@ class MediaSearch(APIView):
                 pagination_class = api_settings.DEFAULT_PAGINATION_CLASS
             paginator = pagination_class()
             page = paginator.paginate_queryset(media, request)
-            serializer = MediaSearchSerializer(
-                page, many=True, context={"request": request}
-            )
+            serializer = MediaSearchSerializer(page, many=True, context={"request": request})
             return paginator.get_paginated_response(serializer.data)
 
 
@@ -806,6 +834,15 @@ class PlaylistList(APIView):
     permission_classes = (permissions.IsAuthenticatedOrReadOnly, IsAuthorizedToAdd)
     parser_classes = (JSONParser, MultiPartParser, FormParser, FileUploadParser)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Playlists'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+        responses={
+            200: openapi.Response('response description', PlaylistSerializer(many=True)),
+        },
+    )
     def get(self, request, format=None):
         pagination_class = api_settings.DEFAULT_PAGINATION_CLASS
         paginator = pagination_class()
@@ -820,6 +857,12 @@ class PlaylistList(APIView):
         serializer = PlaylistSerializer(page, many=True, context={"request": request})
         return paginator.get_paginated_response(serializer.data)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Playlists'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def post(self, request, format=None):
         serializer = PlaylistSerializer(data=request.data, context={"request": request})
         if serializer.is_valid():
@@ -840,15 +883,19 @@ class PlaylistDetail(APIView):
             self.check_object_permissions(self.request, playlist)
             return playlist
         except PermissionDenied:
-            return Response(
-                {"detail": "not enough permissions"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "not enough permissions"}, status=status.HTTP_400_BAD_REQUEST)
         except BaseException:
             return Response(
                 {"detail": "Playlist does not exist"},
                 status=status.HTTP_400_BAD_REQUEST,
             )
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Playlists'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def get(self, request, friendly_token, format=None):
         playlist = self.get_playlist(friendly_token)
         if isinstance(playlist, Response):
@@ -856,31 +903,37 @@ class PlaylistDetail(APIView):
 
         serializer = PlaylistDetailSerializer(playlist, context={"request": request})
 
-        playlist_media = PlaylistMedia.objects.filter(
-            playlist=playlist
-        ).prefetch_related("media__user")
+        playlist_media = PlaylistMedia.objects.filter(playlist=playlist).prefetch_related("media__user")
 
         playlist_media = [c.media for c in playlist_media]
-        playlist_media_serializer = MediaSerializer(
-            playlist_media, many=True, context={"request": request}
-        )
+        playlist_media_serializer = MediaSerializer(playlist_media, many=True, context={"request": request})
         ret = serializer.data
         ret["playlist_media"] = playlist_media_serializer.data
 
         return Response(ret)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Playlists'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def post(self, request, friendly_token, format=None):
         playlist = self.get_playlist(friendly_token)
         if isinstance(playlist, Response):
             return playlist
-        serializer = PlaylistDetailSerializer(
-            playlist, data=request.data, context={"request": request}
-        )
+        serializer = PlaylistDetailSerializer(playlist, data=request.data, context={"request": request})
         if serializer.is_valid():
             serializer.save(user=request.user)
             return Response(serializer.data, status=status.HTTP_201_CREATED)
         return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Playlists'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def put(self, request, friendly_token, format=None):
         playlist = self.get_playlist(friendly_token)
         if isinstance(playlist, Response):
@@ -895,14 +948,10 @@ class PlaylistDetail(APIView):
                 pass
 
         if action in ["add", "remove", "ordering"]:
-            media = Media.objects.filter(
-                friendly_token=media_friendly_token, state="public"
-            ).first()
+            media = Media.objects.filter(friendly_token=media_friendly_token).first()
             if media:
                 if action == "add":
-                    media_in_playlist = PlaylistMedia.objects.filter(
-                        playlist=playlist
-                    ).count()
+                    media_in_playlist = PlaylistMedia.objects.filter(playlist=playlist).count()
                     if media_in_playlist >= settings.MAX_MEDIA_PER_PLAYLIST:
                         return Response(
                             {"detail": "max number of media for a Playlist reached"},
@@ -920,9 +969,7 @@ class PlaylistDetail(APIView):
                             status=status.HTTP_201_CREATED,
                         )
                 elif action == "remove":
-                    PlaylistMedia.objects.filter(
-                        playlist=playlist, media=media
-                    ).delete()
+                    PlaylistMedia.objects.filter(playlist=playlist, media=media).delete()
                     return Response(
                         {"detail": "media removed from Playlist"},
                         status=status.HTTP_201_CREATED,
@@ -935,14 +982,18 @@ class PlaylistDetail(APIView):
                             status=status.HTTP_201_CREATED,
                         )
             else:
-                return Response(
-                    {"detail": "media is not valid"}, status=status.HTTP_400_BAD_REQUEST
-                )
+                return Response({"detail": "media is not valid"}, status=status.HTTP_400_BAD_REQUEST)
         return Response(
             {"detail": "invalid or not specified action"},
             status=status.HTTP_400_BAD_REQUEST,
         )
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Playlists'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def delete(self, request, friendly_token, format=None):
         playlist = self.get_playlist(friendly_token)
         if isinstance(playlist, Response):
@@ -960,6 +1011,7 @@ class EncodingDetail(APIView):
     permission_classes = (permissions.IsAdminUser,)
     parser_classes = (JSONParser, MultiPartParser, FormParser, FileUploadParser)
 
+    @swagger_auto_schema(auto_schema=None)
     def post(self, request, encoding_id):
         ret = {}
         force = request.data.get("force", False)
@@ -992,8 +1044,8 @@ class EncodingDetail(APIView):
                     chunk=chunk,
                     chunk_file_path=chunk_file_path,
                 ).count()
-                > 1
-                and force == False
+                > 1  # noqa
+                and force is False  # noqa
             ):
                 Encoding.objects.filter(id=encoding_id).delete()
                 return Response({"status": "fail"}, status=status.HTTP_400_BAD_REQUEST)
@@ -1013,15 +1065,11 @@ class EncodingDetail(APIView):
             if chunk:
                 original_media_path = chunk_file_path
                 original_media_md5sum = encoding.md5sum
-                original_media_url = (
-                    settings.SSL_FRONTEND_HOST + encoding.media_chunk_url
-                )
+                original_media_url = settings.SSL_FRONTEND_HOST + encoding.media_chunk_url
             else:
                 original_media_path = media.media_file.path
                 original_media_md5sum = media.md5sum
-                original_media_url = (
-                    settings.SSL_FRONTEND_HOST + media.original_media_url
-                )
+                original_media_url = settings.SSL_FRONTEND_HOST + media.original_media_url
 
             ret["original_media_url"] = original_media_url
             ret["original_media_path"] = original_media_path
@@ -1089,6 +1137,7 @@ class EncodingDetail(APIView):
                 return Response({"status": "fail"}, status=status.HTTP_400_BAD_REQUEST)
             return Response({"status": "success"}, status=status.HTTP_201_CREATED)
 
+    @swagger_auto_schema(auto_schema=None)
     def put(self, request, encoding_id, format=None):
         encoding_file = request.data["file"]
         encoding = Encoding.objects.filter(id=encoding_id).first()
@@ -1106,6 +1155,18 @@ class CommentList(APIView):
     permission_classes = (permissions.IsAuthenticatedOrReadOnly, IsAuthorizedToAdd)
     parser_classes = (JSONParser, MultiPartParser, FormParser, FileUploadParser)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='page', type=openapi.TYPE_INTEGER, in_=openapi.IN_QUERY, description='Page number'),
+            openapi.Parameter(name='author', type=openapi.TYPE_STRING, in_=openapi.IN_QUERY, description='username'),
+        ],
+        tags=['Comments'],
+        operation_summary='Lists Comments',
+        operation_description='Paginated listing of all comments',
+        responses={
+            200: openapi.Response('response description', CommentSerializer(many=True)),
+        },
+    )
     def get(self, request, format=None):
         pagination_class = api_settings.DEFAULT_PAGINATION_CLASS
         paginator = pagination_class()
@@ -1137,25 +1198,25 @@ class CommentDetail(APIView):
 
     def get_object(self, friendly_token):
         try:
-            media = Media.objects.select_related("user").get(
-                friendly_token=friendly_token
-            )
+            media = Media.objects.select_related("user").get(friendly_token=friendly_token)
             self.check_object_permissions(self.request, media)
             if media.state == "private" and self.request.user != media.user:
-                return Response(
-                    {"detail": "media is private"}, status=status.HTTP_400_BAD_REQUEST
-                )
+                return Response({"detail": "media is private"}, status=status.HTTP_400_BAD_REQUEST)
             return media
         except PermissionDenied:
-            return Response(
-                {"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST
-            )
+            return Response({"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST)
         except BaseException:
             return Response(
                 {"detail": "media file does not exist"},
                 status=status.HTTP_400_BAD_REQUEST,
             )
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Media'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def get(self, request, friendly_token):
         # list comments for a media
         media = self.get_object(friendly_token)
@@ -1168,6 +1229,12 @@ class CommentDetail(APIView):
         serializer = CommentSerializer(page, many=True, context={"request": request})
         return paginator.get_paginated_response(serializer.data)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Media'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def delete(self, request, friendly_token, uid=None):
         """Delete a comment
         Administrators, MediaCMS editors and managers,
@@ -1181,18 +1248,18 @@ class CommentDetail(APIView):
                     {"detail": "comment does not exist"},
                     status=status.HTTP_400_BAD_REQUEST,
                 )
-            if (
-                (comment.user == self.request.user)
-                or comment.media.user == self.request.user
-                or is_mediacms_editor(self.request.user)
-            ):
+            if (comment.user == self.request.user) or comment.media.user == self.request.user or is_mediacms_editor(self.request.user):
                 comment.delete()
             else:
-                return Response(
-                    {"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST
-                )
+                return Response({"detail": "bad permissions"}, status=status.HTTP_400_BAD_REQUEST)
         return Response(status=status.HTTP_204_NO_CONTENT)
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Media'],
+        operation_summary='to_be_written',
+        operation_description='to_be_written',
+    )
     def post(self, request, friendly_token):
         """Create a comment"""
         media = self.get_object(friendly_token)
@@ -1217,17 +1284,19 @@ class CommentDetail(APIView):
 class UserActions(APIView):
     parser_classes = (JSONParser,)
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='action', type=openapi.TYPE_STRING, in_=openapi.IN_PATH, description='action', required=True, enum=VALID_USER_ACTIONS),
+        ],
+        tags=['Users'],
+        operation_summary='List user actions',
+        operation_description='Lists user actions',
+    )
     def get(self, request, action):
         media = []
         if action in VALID_USER_ACTIONS:
             if request.user.is_authenticated:
-                media = (
-                    Media.objects.select_related("user")
-                    .filter(
-                        mediaactions__user=request.user, mediaactions__action=action
-                    )
-                    .order_by("-mediaactions__action_date")
-                )
+                media = Media.objects.select_related("user").filter(mediaactions__user=request.user, mediaactions__action=action).order_by("-mediaactions__action_date")
             elif request.session.session_key:
                 media = (
                     Media.objects.select_related("user")
@@ -1248,11 +1317,18 @@ class UserActions(APIView):
 class CategoryList(APIView):
     """List categories"""
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Categories'],
+        operation_summary='Lists Categories',
+        operation_description='Lists all categories',
+        responses={
+            200: openapi.Response('response description', CategorySerializer),
+        },
+    )
     def get(self, request, format=None):
         categories = Category.objects.filter().order_by("title")
-        serializer = CategorySerializer(
-            categories, many=True, context={"request": request}
-        )
+        serializer = CategorySerializer(categories, many=True, context={"request": request})
         ret = serializer.data
         return Response(ret)
 
@@ -1260,6 +1336,17 @@ class CategoryList(APIView):
 class TagList(APIView):
     """List tags"""
 
+    @swagger_auto_schema(
+        manual_parameters=[
+            openapi.Parameter(name='page', type=openapi.TYPE_INTEGER, in_=openapi.IN_QUERY, description='Page number'),
+        ],
+        tags=['Tags'],
+        operation_summary='Lists Tags',
+        operation_description='Paginated listing of all tags',
+        responses={
+            200: openapi.Response('response description', TagSerializer),
+        },
+    )
     def get(self, request, format=None):
         tags = Tag.objects.filter().order_by("-media_count")
         pagination_class = api_settings.DEFAULT_PAGINATION_CLASS
@@ -1272,17 +1359,24 @@ class TagList(APIView):
 class EncodeProfileList(APIView):
     """List encode profiles"""
 
+    @swagger_auto_schema(
+        manual_parameters=[],
+        tags=['Encoding Profiles'],
+        operation_summary='List Encoding Profiles',
+        operation_description='Lists all encoding profiles for videos',
+        responses={200: EncodeProfileSerializer(many=True)},
+    )
     def get(self, request, format=None):
         profiles = EncodeProfile.objects.all()
-        serializer = EncodeProfileSerializer(
-            profiles, many=True, context={"request": request}
-        )
+        serializer = EncodeProfileSerializer(profiles, many=True, context={"request": request})
         return Response(serializer.data)
 
 
 class TasksList(APIView):
     """List tasks"""
 
+    swagger_schema = None
+
     permission_classes = (permissions.IsAdminUser,)
 
     def get(self, request, format=None):
@@ -1293,6 +1387,8 @@ class TasksList(APIView):
 class TaskDetail(APIView):
     """Cancel a task"""
 
+    swagger_schema = None
+
     permission_classes = (permissions.IsAdminUser,)
 
     def delete(self, request, uid, format=None):

frontend/.babelrc

@@ -0,0 +1,12 @@
+{
+	"presets": [
+		"@babel/react", ["@babel/env", {
+			"modules": false,
+			"useBuiltIns": "usage",
+    		"corejs": 3,
+			"targets": {
+				"browsers": ["defaults"]
+			}
+		}]
+	]
+}

frontend/.env

@@ -0,0 +1,12 @@
+MEDIACMS_ID=mediacms-frontend
+MEDIACMS_TITLE=MediaCMS Frontend
+
+MEDIACMS_URL=http://localhost
+MEDIACMS_API=http://localhost/api/v1
+
+MEDIACMS_USER_IS_ADMIN=true
+MEDIACMS_USER_IS_ANONYMOUS=false
+
+MEDIACMS_USER_USERNAME=admin
+MEDIACMS_USER_NAME=Admin
+MEDIACMS_USER_THUMB=http://localhost/media/userlogos/user.jpg

frontend/.gitignore

@@ -0,0 +1,131 @@
+.DS_Store
+
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+lerna-debug.log*
+
+# Diagnostic reports (https://nodejs.org/api/report.html)
+report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
+
+# Runtime data
+pids
+*.pid
+*.seed
+*.pid.lock
+
+# Directory for instrumented libs generated by jscoverage/JSCover
+lib-cov
+
+# Coverage directory used by tools like istanbul
+coverage
+*.lcov
+
+# nyc test coverage
+.nyc_output
+
+# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
+.grunt
+
+# Bower dependency directory (https://bower.io/)
+bower_components
+
+# node-waf configuration
+.lock-wscript
+
+# Compiled binary addons (https://nodejs.org/api/addons.html)
+build/Release
+
+# Dependency directories
+node_modules/
+jspm_packages/
+
+# Snowpack dependency directory (https://snowpack.dev/)
+web_modules/
+
+# TypeScript cache
+*.tsbuildinfo
+
+# Optional npm cache directory
+.npm
+
+# Optional eslint cache
+.eslintcache
+
+# Microbundle cache
+.rpt2_cache/
+.rts2_cache_cjs/
+.rts2_cache_es/
+.rts2_cache_umd/
+
+# Optional REPL history
+.node_repl_history
+
+# Output of 'npm pack'
+*.tgz
+!modules/**/*.tgz
+
+# Yarn Integrity file
+.yarn-integrity
+
+# dotenv environment variables file
+.env.development
+
+# parcel-bundler cache (https://parceljs.org/)
+.cache
+.parcel-cache
+
+# Next.js build output
+.next
+out
+
+# Nuxt.js build / generate output
+.nuxt
+# dist
+
+# Gatsby files
+.cache/
+# Comment in the public line in if your project uses Gatsby and not Next.js
+# https://nextjs.org/blog/next-9-1#public-directory-support
+# public
+
+# vuepress build output
+.vuepress/dist
+
+# Serverless directories
+.serverless/
+
+# FuseBox cache
+.fusebox/
+
+# DynamoDB Local files
+.dynamodb/
+
+# TernJS port file
+.tern-port
+
+# Stores VSCode versions used for testing VSCode extensions
+.vscode-test
+
+# yarn v2
+.yarn/cache
+.yarn/unplugged
+.yarn/build-state.yml
+.yarn/install-state.gz
+.pnp.*
+
+# System files
+.fuse_hidden*
+
+# Exports
+/build
+/dist
+
+# Packages dev files
+# /packages/**/package-lock.json
+
+# Other
+*playground*

frontend/.prettierignore

@@ -0,0 +1 @@
+config/templates/*.ejs

frontend/.prettierrc

@@ -0,0 +1,5 @@
+{
+  "semi": true,
+  "singleQuote": true,
+  "printWidth": 120
+}

frontend/.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+    "editor.formatOnSave": true
+}

frontend/README.md

@@ -0,0 +1,29 @@
+# MediaCMS Web Client (demo)
+
+### **Requirements**
+
+- nodejs: version >= 14.17.0
+
+---
+
+### **Installation**
+
+    npm install
+
+---
+
+### **Development**
+
+    npm run start
+
+Open in browser: [http://localhost:8088](http://localhost:8088)
+
+---
+
+### **Build**
+
+    npm run dist
+
+Generates the folder "**_frontend/dist_**".
+
+Copy folders and files from "**_frontend/dist/static_**" into "**_static_**".

frontend/config/mediacms.config.html.js

@@ -0,0 +1,44 @@
+module.exports = {
+  head: {
+    meta: [
+      { charset: 'utf-8' },
+      { content: 'ie=edge', 'http-equiv': 'x-ua-compatible' },
+      { name: 'viewport', content: 'width=device-width, initial-scale=1' },
+      { name: 'theme-color', content: '#fafafa' },
+      { name: 'msapplication-TileColor', content: '#fafafa' },
+      { name: 'msapplication-config', content: 'favicons/browserconfig.xml' },
+    ],
+    links: [
+      /**
+       * Manifest file link.
+       */
+      { rel: 'manifest', href: 'static/favicons/site.webmanifest' },
+      /**
+       * Favicon links.
+       */
+      { rel: 'apple-touch-icon', sizes: '180x180', href: 'static/favicons/apple-touch-icon.png' },
+      { rel: 'icon', type: 'image/png', sizes: '32x32', href: 'static/favicons/favicon-32x32.png' },
+      { rel: 'icon', type: 'image/png', sizes: '16x16', href: 'static/favicons/favicon-16x16.png' },
+      { rel: 'mask-icon', href: 'static/favicons/safari-pinned-tab.svg', color: '#fafafa' },
+      { rel: 'shortcut icon', href: 'static/favicons/favicon.ico' },
+      /**
+       * Stylesheet links
+       */
+      { rel: 'preload', href: 'static/css/_extra.css', as: 'style' },
+      { rel: 'stylesheet', href: 'static/css/_extra.css' },
+
+      // 'https://fonts.googleapis.com/icon?family=Material+Icons',
+      { rel: 'preload', href: 'static/lib/material-icons/material-icons.css', as: 'style' },
+      { rel: 'stylesheet', href: 'static/lib/material-icons/material-icons.css' },
+
+      // 'https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,500,500i,700,700i&display=swap
+      { rel: 'preload', href: 'static/lib/gfonts/gfonts.css', as: 'style' },
+      { rel: 'stylesheet', href: 'static/lib/gfonts/gfonts.css' },
+    ],
+    scripts: [],
+  },
+  body: {
+    scripts: [],
+    snippet: '',
+  },
+};

frontend/config/mediacms.config.js

@@ -0,0 +1,31 @@
+const path = require('path');
+
+const rootDir = '..';
+const srcDir = rootDir + '/src';
+const configDir = srcDir + '/templates/config';
+const coreConfigDir = configDir + '/core';
+const installConfigDir = configDir + '/installation';
+
+module.exports = {
+  src: path.resolve(__dirname, srcDir),
+  build: path.resolve(__dirname, rootDir),
+  html: require('./mediacms.config.html.js'),
+  pages: require('./mediacms.config.pages.js'),
+  window: {
+    MediaCMS: {
+      api: require(coreConfigDir + '/api.config.js'),
+      url: require(coreConfigDir + '/url.config.js'),
+      user: require(coreConfigDir + '/user.config.js'),
+      site: require(installConfigDir + '/site.config.js'),
+      pages: require(installConfigDir + '/pages.config.js'),
+      contents: require(installConfigDir + '/contents.config.js'),
+      features: require(installConfigDir + '/features.config.js'),
+      /*notifications: [
+          'Message one text',
+          'Message two text',
+          'Message three text'
+      ],*/
+    },
+  },
+  postcssConfigFile: path.resolve(__dirname, './postcss.config.js'),
+};

frontend/config/mediacms.config.pages.js

@@ -0,0 +1,268 @@
+const templates = require('./mediacms.config.templates');
+
+const DEV_SAMPLE_DATA = {
+  profileId: process.env.MEDIACMS_USER_USERNAME,
+  media: {
+    videoId: process.env.MEDIACMS_VIDEO_ID,
+    audioId: process.env.MEDIACMS_AUDIO_ID,
+    imageId: process.env.MEDIACMS_IMAGE_ID,
+    pdfId: process.env.MEDIACMS_PDF_ID,
+  },
+  playlistId: process.env.MEDIACMS_PLAYLIST_ID,
+};
+
+const formatPage = (page) => {
+  const pageContentId = 'page-' + page.id;
+  const filename = page.filename ? page.filename : ('home' === page.id ? 'index' : page.id) + '.html';
+  const render = page.renderer
+    ? page.renderer
+    : page.component
+    ? templates.renderPageContent({ page: { id: pageContentId, component: page.component } })
+    : undefined;
+  const headLinks = [
+    { rel: 'preload', href: './static/lib/video-js/7.7.5/video.min.js', as: 'script' },
+    ...(page.headLinks ? page.headLinks : []),
+  ];
+  const bodyScripts = [
+    { src: './static/lib/video-js/7.7.5/video.min.js' },
+    ...(page.bodyScripts ? page.bodyScripts : []),
+  ];
+
+  const ret = {
+    buildExclude: !!page.buildExclude,
+    title: page.title,
+    filename,
+    render,
+    html: {
+      head: {
+        links: headLinks,
+      },
+      body: {
+        scripts: bodyScripts,
+        snippet: page.snippet || templates.htmlBodySnippet({ id: pageContentId }),
+      },
+    },
+    window: { MediaCMS: page.global ? { ...page.global } : {} },
+  };
+
+  return ret;
+};
+
+const formatPageData = (page) => {
+  return formatPage({
+    ...page,
+  });
+};
+
+const formatStaticPageData = (page) => {
+  const pageContentId = 'page-' + page.id;
+  return formatPage({
+    ...page,
+    renderer: page.renderer
+      ? page.renderer
+      : page.component
+      ? templates.renderPageStaticContent({ page: { id: pageContentId, component: page.component } })
+      : undefined,
+  });
+};
+
+const PAGES = {
+  base: {
+    id: 'base',
+    title: 'Layout base',
+    renderer: templates.renderBase(),
+  },
+  index: { id: 'home', title: 'Home', component: 'HomePage' },
+  search: { id: 'search', title: 'Search results', component: 'SearchPage' },
+  latest: { id: 'latest', title: 'Recent uploads', component: 'LatestMediaPage' },
+  featured: { id: 'featured', title: 'Featured', component: 'FeaturedMediaPage' },
+  recommended: { id: 'recommended', title: 'Recommended', component: 'RecommendedMediaPage' },
+  members: { id: 'members', title: 'Members', component: 'MembersPage' },
+  history: { id: 'history', title: 'History', component: 'HistoryPage' },
+  liked: { id: 'liked', title: 'Liked media', component: 'LikedMediaPage' },
+  tags: { id: 'tags', title: 'Tags', component: 'TagsPage' },
+  categories: { id: 'categories', title: 'Categories', component: 'CategoriesPage' },
+  'manage-media': { id: 'manage-media', title: 'Manage media', component: 'ManageMediaPage' },
+  'manage-users': { id: 'manage-users', title: 'Manage users', component: 'ManageUsersPage' },
+  'manage-comments': { id: 'manage-comments', title: 'Manage comments', component: 'ManageCommentsPage' },
+  'add-media': {
+    id: 'add-media',
+    title: 'Add media',
+    renderer: templates.renderAddMediaPageContent(),
+    snippet: templates.htmlBodySnippetAddMediaPage(),
+    headLinks: [{ rel: 'preload', href: './static/lib/file-uploader/5.13.0/fine-uploader.min.js', as: 'script' }],
+    bodyScripts: [{ src: './static/lib/file-uploader/5.13.0/fine-uploader.min.js' }],
+  },
+  embed: {
+    id: 'embed',
+    title: 'Embedded player',
+    renderer: templates.renderEmbedPageContent({ page: { id: 'page-embed', component: 'EmbedPage' } }),
+    snippet: templates.htmlBodySnippetEmbedPage({ id: 'page-embed' }),
+    global: { mediaId: DEV_SAMPLE_DATA.media.videoId },
+  },
+  media: {
+    id: 'media',
+    title: 'Media',
+    component: 'MediaPage',
+    global: { mediaId: DEV_SAMPLE_DATA.media.videoId },
+  },
+  'media-video': {
+    buildExclude: true,
+    id: 'media-video',
+    title: 'Media - Video',
+    component: 'MediaVideoPage',
+    global: { mediaId: DEV_SAMPLE_DATA.media.videoId },
+  },
+  'media-audio': {
+    buildExclude: true,
+    id: 'media-audio',
+    title: 'Media - Audio',
+    component: 'MediaAudioPage',
+    global: { mediaId: DEV_SAMPLE_DATA.media.audioId },
+  },
+  'media-image': {
+    buildExclude: true,
+    id: 'media-image',
+    title: 'Media - Image',
+    component: 'MediaImagePage',
+    global: { mediaId: DEV_SAMPLE_DATA.media.imageId },
+  },
+  'media-pdf': {
+    buildExclude: true,
+    id: 'media-pdf',
+    title: 'Media - Pdf',
+    component: 'MediaPdfPage',
+    global: { mediaId: DEV_SAMPLE_DATA.media.pdfId },
+  },
+  playlist: {
+    id: 'playlist',
+    title: 'Playlist',
+    component: 'PlaylistPage',
+    global: { playlistId: DEV_SAMPLE_DATA.playlistId },
+  },
+  'profile-media': {
+    id: 'profile-media',
+    title: 'Profile - Media',
+    component: 'ProfileMediaPage',
+    global: { profileId: DEV_SAMPLE_DATA.profileId },
+  },
+  'profile-about': {
+    id: 'profile-about',
+    title: 'Profile - About',
+    component: 'ProfileAboutPage',
+    global: { profileId: DEV_SAMPLE_DATA.profileId },
+  },
+  'profile-playlists': {
+    id: 'profile-playlists',
+    title: 'Profile - Playlist',
+    component: 'ProfilePlaylistsPage',
+    global: { profileId: DEV_SAMPLE_DATA.profileId },
+  },
+};
+
+const STATIC_PAGES = {
+  error: {
+    buildExclude: true,
+    id: 'error',
+    title: 'Error',
+    renderer: templates.renderBase(),
+    snippet: templates.static.errorPage(),
+  },
+  about: {
+    id: 'about',
+    title: 'About',
+    renderer: templates.renderBase(),
+    snippet: templates.static.aboutPage(),
+  },
+  terms: {
+    buildExclude: true,
+    id: 'terms',
+    title: 'Terms',
+    renderer: templates.renderBase(),
+    snippet: templates.static.termsPage(),
+  },
+};
+
+const DEV_ONLY_STATIC_PAGES = {
+  'add-media-template': {
+    buildExclude: true,
+    id: 'add-media-template',
+    title: 'Add media - Template',
+    renderer: templates.renderAddMediaPageContent(),
+    snippet: templates.static.addMediaPageTemplate(),
+    headLinks: [{ rel: 'preload', href: './static/lib/file-uploader/5.13.0/fine-uploader.min.js', as: 'script' }],
+    bodyScripts: [{ src: './static/lib/file-uploader/5.13.0/fine-uploader.min.js' }],
+  },
+  'edit-media': {
+    buildExclude: true,
+    id: 'edit-media',
+    title: 'Edit media',
+    renderer: templates.renderBase(),
+    snippet: templates.static.editMediaPage(),
+  },
+  'edit-channel': {
+    buildExclude: true,
+    id: 'edit-channel',
+    title: 'Edit channel',
+    renderer: templates.renderBase(),
+    snippet: templates.static.editChannelPage(),
+  },
+  'edit-profile': {
+    buildExclude: true,
+    id: 'edit-profile',
+    title: 'Edit profile',
+    renderer: templates.renderBase(),
+    snippet: templates.static.editProfilePage(),
+  },
+  signin: {
+    buildExclude: true,
+    id: 'signin',
+    title: 'Sign in',
+    renderer: templates.renderBase(),
+    snippet: templates.static.signinPage(),
+  },
+  signout: {
+    buildExclude: true,
+    id: 'signout',
+    title: 'Sign out',
+    renderer: templates.renderBase(),
+    snippet: templates.static.signoutPage(),
+  },
+  register: {
+    buildExclude: true,
+    id: 'register',
+    title: 'Register',
+    renderer: templates.renderBase(),
+    snippet: templates.static.registerPage(),
+  },
+  'reset-password': {
+    buildExclude: true,
+    id: 'reset-password',
+    title: 'Reset password',
+    renderer: templates.renderBase(),
+    snippet: templates.static.resetPasswordPage(),
+  },
+  contact: {
+    buildExclude: true,
+    id: 'contact',
+    title: 'Contact us',
+    renderer: templates.renderBase(),
+    snippet: templates.static.contactPage(),
+  },
+};
+
+const pages = {};
+
+for (let k in PAGES) {
+  pages[k] = formatPageData(PAGES[k]);
+}
+
+for (let k in STATIC_PAGES) {
+  pages[k] = formatStaticPageData(STATIC_PAGES[k]);
+}
+
+for (let k in DEV_ONLY_STATIC_PAGES) {
+  pages[k] = formatStaticPageData(DEV_ONLY_STATIC_PAGES[k]);
+}
+
+module.exports = pages;

frontend/config/mediacms.config.templates.js

@@ -0,0 +1,45 @@
+const fs = require('fs');
+const path = require('path');
+const ejs = require('ejs');
+
+const templatesPath = path.join(__dirname, './templates');
+const staticTemplatesPath = path.join(__dirname, './templates/static');
+
+const compileTmpl = (filename) =>
+  ejs.compile(fs.readFileSync(path.join(templatesPath, filename), 'utf8'), {
+    root: [templatesPath],
+    filename: path.join(templatesPath, filename),
+    outputFunctionName: 'echo',
+  });
+
+const compileStaticTmpl = (filename) =>
+  ejs.compile(fs.readFileSync(path.join(staticTemplatesPath, filename), 'utf8'), {
+    root: [staticTemplatesPath],
+    filename: path.join(staticTemplatesPath, filename),
+    outputFunctionName: 'echo',
+  });
+
+module.exports = {
+  htmlBodySnippet: compileTmpl('htmlBodySnippet.ejs'),
+  htmlBodySnippetEmbedPage: compileTmpl('htmlBodySnippetEmbedPage.ejs'),
+  htmlBodySnippetAddMediaPage: compileTmpl('htmlBodySnippetAddMediaPage.ejs'),
+  renderBase: compileTmpl('renderBase.ejs'),
+  renderPageContent: compileTmpl('renderPageContent.ejs'),
+  renderPageStaticContent: compileTmpl('renderPageStaticContent.ejs'),
+  renderEmbedPageContent: compileTmpl('renderEmbedPageContent.ejs'),
+  renderAddMediaPageContent: compileTmpl('renderAddMediaPageContent.ejs'),
+  static: {
+    errorPage: compileStaticTmpl('errorPage.html'),
+    aboutPage: compileStaticTmpl('aboutPage.html'),
+    termsPage: compileStaticTmpl('termsPage.html'),
+    contactPage: compileStaticTmpl('contactPage.html'),
+    signinPage: compileStaticTmpl('signinPage.html'),
+    signoutPage: compileStaticTmpl('signoutPage.html'),
+    registerPage: compileStaticTmpl('registerPage.html'),
+    resetPasswordPage: compileStaticTmpl('resetPasswordPage.html'),
+    editMediaPage: compileStaticTmpl('editMediaPage.html'),
+    editChannelPage: compileStaticTmpl('editChannelPage.html'),
+    editProfilePage: compileStaticTmpl('editProfilePage.html'),
+    addMediaPageTemplate: compileStaticTmpl('addMediaPageTemplate.html'),
+  },
+};

frontend/config/postcss.config.js

@@ -0,0 +1,10 @@
+module.exports = (ctx) => {
+  const ret = {
+    map: ctx.env === 'development' ? ctx.map : false,
+    plugins: {
+      autoprefixer: {},
+    },
+  };
+
+  return ret;
+};

frontend/config/templates/htmlBodySnippet.ejs

@@ -0,0 +1,6 @@
+<div id="app-header"></div>
+<div id="app-sidebar"></div>
+<div class="page-main-wrap">
+    <% if (id) { %><div id="<%= id %>"></div><% } %>
+</div>
+<div id="app-footer"></div>

frontend/config/templates/htmlBodySnippetAddMediaPage.ejs

@@ -0,0 +1,65 @@
+<div id="add-media-page">
+	
+	<div class="page-container">
+	
+		<div id="app-header"></div>
+		<div id="app-sidebar"></div>
+
+		<div class="page-main-wrap">
+		
+			<main class="page-main">
+				<div class="page-main-inner">
+
+					<div class="media-uploader-wrap">
+						
+						<div class="media-uploader-top-wrap">
+							
+							<div class="media-uploader-top-left-wrap">
+								<h1>Upload media files</h1>
+							</div>
+							
+							<div class="media-uploader-top-right-wrap"></div>
+						</div>
+
+						<script type="text/template" id="qq-template"> <div class="media-uploader-bottom-wrap qq-uploader-selector"> <div class="media-uploader-bottom-left-wrap"> <div class="media-drag-drop-wrap"> <div class="media-drag-drop-inner" qq-drop-area-text="Drop files here"> <div class="media-drag-drop-content"> <div class="media-drag-drop-content-inner"> <span><i class="material-icons">cloud_upload</i></span> <span>Drag and drop files</span> <span>or</span> <span class="browse-files-btn-wrap"><span class="qq-upload-button-selector">Browse your files</span></span><div class="qq-upload-drop-area-selector media-dropzone" qq-hide-dropzone><span class="qq-upload-drop-area-text-selector"></span></div> </div></div></div></div></div><div class="media-uploader-bottom-right-wrap"> <ul class="media-upload-items-list qq-upload-list-selector"> <li> <div class="media-upload-item-main"> <div class="media-upload-item-thumb"> <img class="qq-thumbnail-selector" qq-max-size="120" qq-server-scale alt=""/> <span class="media-upload-item-spinner qq-upload-spinner-selector"><i class="material-icons">autorenew</i></span> <button type="button" class="qq-upload-retry-selector retry-media-upload-item" aria-label="Retry"><i class="material-icons">refresh</i> Retry</button> </div><div class="media-upload-item-details"> <div class="media-upload-item-name"> <span class="media-upload-item-filename qq-upload-file-selector"></span> <input class="media-upload-item-filename-input qq-edit-filename-selector" tab-index="0" type="text"/> </div><div class="media-upload-item-details-bottom"> <div class="media-upload-item-progress-bar-container qq-progress-bar-container-selector"> <div role="progressbar" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100" class="media-upload-item-progress-bar qq-progress-bar-selector"></div></div><span class="media-upload-item-upload-size qq-upload-size-selector"></span> <span role="status" class="media-upload-item-status-text qq-upload-status-text-selector"></span> </div><div class="media-upload-item-top-actions"> <span class="filename-edit qq-edit-filename-icon-selector" aria-label="Edit filename">Edit filename <i class="material-icons">create</i></span> <button type="button" class="delete-media-upload-item qq-upload-delete-selector" aria-label="Delete">Delete <i class="material-icons">delete</i></button> <button type="button" class="cancel-media-upload-item qq-upload-cancel-selector" aria-label="Cancel">Cancel <i class="material-icons">cancel</i></button> </div><div class="media-upload-item-bottom-actions"> <button type="button" class="continue-media-upload-item qq-upload-continue-selector" aria-label="Continue"><i class="material-icons">play_circle_outline</i> Continue</button> <button type="button" class="pause-media-upload-item qq-upload-pause-selector" aria-label="Pause"><i class="material-icons">pause_circle_outline</i> Pause</button> </div></div></div></li></ul> <dialog class="qq-alert-dialog-selector"> <div class="qq-dialog-message-selector"></div><div class="qq-dialog-buttons"> <button type="button" class="qq-cancel-button-selector">Close</button> </div></dialog> <dialog class="qq-confirm-dialog-selector"> <div class="qq-dialog-message-selector"></div><div class="qq-dialog-buttons"> <button type="button" class="qq-cancel-button-selector">No</button> <button type="button" class="qq-ok-button-selector">Yes</button> </div></dialog> <dialog class="qq-prompt-dialog-selector"> <div class="qq-dialog-message-selector"></div><input type="text"> <div class="qq-dialog-buttons"> <button type="button" class="qq-cancel-button-selector">Cancel</button> <button type="button" class="qq-ok-button-selector">Ok</button> </div></dialog> </div></div></script>
+
+						<div class="media-uploader"></div>
+					</div>
+
+				</div>
+			</main>
+
+		</div>
+	
+	</div>
+
+</div>
+<script type='text/javascript'>
+
+	function csrfToken(){var a,b=null;if(document.cookie&&""!==document.cookie){var c=document.cookie.split(";");for(a=0;a<c.length;){var d=c[a].trim();if("csrftoken="===d.substring(0,10)){b=decodeURIComponent(d.substring(10));break}a+=1}}return b};
+
+	document.addEventListener("DOMContentLoaded", function(event) {
+	    var default_concurrent_chunked_uploader = new qq.FineUploader({
+	        debug: true,
+	        element: document.querySelector('.media-uploader'),
+	        request: {
+	            endpoint: '/fu/upload/',
+	            customHeaders: {
+	                'X-CSRFToken': csrfToken('csrftoken'),
+	            },
+	        },
+	        retry: {
+	            enableAuto: true,
+	        },
+	        chunking: {
+	            enabled: true,
+	            concurrent: {
+	                enabled: true,
+	            },
+	            success: {
+	                endpoint: '/fu/upload/?done',
+	            },
+	        },
+	    });
+	});
+</script>

frontend/config/templates/htmlBodySnippetEmbedPage.ejs

@@ -0,0 +1,3 @@
+<div class="page-main-wrap">
+<% if (id) { %><div id="<%= id %>"></div><% } %>
+</div>