Compare commits

..

No commits in common. "main" and "v1.2" have entirely different histories.
main ... v1.2

1281 changed files with 72286 additions and 151947 deletions

View File

@ -1,4 +0,0 @@
[run]
omit =
*bento4*
*/migrations/*

View File

@ -1,2 +0,0 @@
node_modules
npm-debug.log

3
.github/FUNDING.yml vendored
View File

@ -1,3 +0,0 @@
# These are supported funding model platforms
github: [mgogoulos]

View File

@ -1,31 +0,0 @@
---
name: Issue report
about: Create a report to help us improve MediaCMS
title: ''
labels: 'issue: bug'
assignees: mgogoulos
---
**Describe the issue**
A clear and concise description of what the issue is.
**To Reproduce**
Steps to reproduce the issue:
1. Go to ...
2. Perform action ...
3. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Environment (please complete the following information):**
- OS: [e.g. Ubuntu Linux]
- Installation method: [Docker install, or single server install]
- Browser, if applicable
**Additional context**
Add any other context about the problem here.

View File

@ -1,17 +0,0 @@
---
name: Feature request
about: Suggest an idea
title: ''
labels: 'issue: enhancement'
assignees: mgogoulos
---
**Describe the feature you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@ -1,10 +0,0 @@
## Description
<!-- Describe the changes introduced by this PR for the reviewers to fully understand. -->
## Steps
<!-- Actions to be done pre and post deployment -->
*Pre-deploy*
*Post-deploy*

View File

@ -1,20 +0,0 @@
---
name: "CI"
on:
pull_request:
push:
branches:
- main
paths-ignore:
- '**/README.md'
jobs:
pre-commit:
uses: ./.github/workflows/pre-commit.yml
test:
uses: ./.github/workflows/python.yml
needs: [pre-commit]
release:
uses: ./.github/workflows/docker-build-push.yml
secrets: inherit # pass all secrets
needs: [test]
if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'

View File

@ -1,78 +0,0 @@
name: Docker build and push
on:
workflow_call:
push:
tags:
- v*.*.*
jobs:
release:
name: Build & release to DockerHub
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Login to Docker Hub
uses: docker/login-action@v2.2.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Docker meta for base image
id: meta-base
uses: docker/metadata-action@v4
with:
images: |
mediacms/mediacms
tags: |
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
labels: |
org.opencontainers.image.title=MediaCMS
org.opencontainers.image.description=MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.
org.opencontainers.image.vendor=MediaCMS
org.opencontainers.image.url=https://mediacms.io/
org.opencontainers.image.source=https://github.com/mediacms-io/mediacms
org.opencontainers.image.licenses=AGPL-3.0
- name: Docker meta for full image
id: meta-full
uses: docker/metadata-action@v4
with:
images: |
mediacms/mediacms
tags: |
type=raw,value=full,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
type=semver,pattern={{version}}-full
type=semver,pattern={{major}}.{{minor}}-full
type=semver,pattern={{major}}-full
labels: |
org.opencontainers.image.title=MediaCMS Full
org.opencontainers.image.description=MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API. This is the full version with additional dependencies.
org.opencontainers.image.vendor=MediaCMS
org.opencontainers.image.url=https://mediacms.io/
org.opencontainers.image.source=https://github.com/mediacms-io/mediacms
org.opencontainers.image.licenses=AGPL-3.0
- name: Build and push full image
uses: docker/build-push-action@v4
with:
context: .
target: full
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta-full.outputs.tags }}
labels: ${{ steps.meta-full.outputs.labels }}
- name: Build and push base image
uses: docker/build-push-action@v4
with:
context: .
target: base
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta-base.outputs.tags }}
labels: ${{ steps.meta-base.outputs.labels }}

View File

@ -1,15 +0,0 @@
name: pre-commit
on:
workflow_call:
jobs:
pre-commit:
name: Pre-Commit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
- uses: pre-commit/action@v3.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}

View File

@ -1,35 +0,0 @@
name: Python Tests
on:
workflow_call:
jobs:
build:
name: Build & test via docker-compose
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Build the Stack
run: docker compose -f docker-compose-dev.yaml build
- name: Start containers
run: docker compose -f docker-compose-dev.yaml up -d
- name: List containers
run: docker ps
- name: Sleep for 60 seconds
run: sleep 60s
shell: bash
- name: Run Django Tests
run: docker compose -f docker-compose-dev.yaml exec --env TESTING=True -T web pytest
# Run with coverage, saves report on htmlcov dir
# run: docker-compose -f docker-compose-dev.yaml exec --env TESTING=True -T web pytest --cov --cov-report=html --cov-config=.coveragerc
- name: Tear down the Stack
run: docker compose -f docker-compose-dev.yaml down

22
.gitignore vendored
View File

@ -1,11 +1,8 @@
cli-tool/.env
frontend/package-lock.json
media_files/encoded/ media_files/encoded/
media_files/original/ media_files/original/
media_files/hls/ media_files/hls/
media_files/chunks/ media_files/chunks/
media_files/uploads/ media_files/uploads/
media_files/tinymce_media/
postgres_data/ postgres_data/
celerybeat-schedule celerybeat-schedule
logs/ logs/
@ -15,23 +12,4 @@ static/ckeditor/
static/debug_toolbar/ static/debug_toolbar/
static/mptt/ static/mptt/
static/rest_framework/ static/rest_framework/
static/drf-yasg
cms/local_settings.py cms/local_settings.py
deploy/docker/local_settings.py
yt.readme.md
/frontend-tools/video-editor/node_modules
/frontend-tools/video-editor/client/node_modules
/static_collected
/frontend-tools/video-editor-v1
frontend-tools/.DS_Store
static/video_editor/videos/sample-video-30s.mp4
static/video_editor/videos/sample-video-37s.mp4
/frontend-tools/video-editor-v2
.DS_Store
static/video_editor/videos/sample-video-10m.mp4
static/video_editor/videos/sample-video-10s.mp4
frontend-tools/video-js/public/videos/sample-video-white.mp4
frontend-tools/video-editor/client/public/videos/sample-video.mp3
frontend-tools/chapters-editor/client/public/videos/sample-video.mp3
static/chapters_editor/videos/sample-video.mp3
static/video_editor/videos/sample-video.mp3

1
.mailmap Normal file
View File

@ -0,0 +1 @@
Swift Ugandan <swiftugandan@gmail.com> <swiftugandan@gmail.com>

View File

@ -1,16 +0,0 @@
repos:
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
hooks:
- id: flake8
- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
args: ["--profile", "black"]
- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
- id: black
language_version: python3
additional_dependencies: [ 'click==8.0.4' ]

View File

@ -1,3 +0,0 @@
/templates/cms/*
/templates/*.html
*.scss

View File

@ -1,21 +0,0 @@
{
"semi": true,
"singleQuote": true,
"printWidth": 120,
"tabWidth": 4,
"useTabs": false,
"trailingComma": "es5",
"bracketSpacing": true,
"bracketSameLine": false,
"arrowParens": "always",
"endOfLine": "lf",
"embeddedLanguageFormatting": "auto",
"overrides": [
{
"files": ["*.css", "*.scss"],
"options": {
"singleQuote": false
}
}
]
}

View File

@ -1 +1,8 @@
Please see https://github.com/mediacms-io/mediacms/graphs/contributors for complete list of contributors to this repository! Wordgames.gr - https://www.wordgames.gr
Yiannis Stergiou - ys.stergiou@gmail.com
Markos Gogoulos - mgogoulos@gmail.com
Contributors
Swift Ugandan - swiftugandan@gmail.com

View File

@ -1,13 +0,0 @@
# Contributor Code of Conduct
As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
This Code of Conduct is adapted from the [Contributor Covenant](http:contributor-covenant.org), version 1.0.0, available at https://www.contributor-covenant.org/version/1/0/0/code-of-conduct.html

View File

@ -1,113 +1,66 @@
FROM python:3.13.5-slim-bookworm AS build-image FROM python:3.8-buster AS compile-image
# Install system dependencies needed for downloading and extracting SHELL ["/bin/bash", "-c"]
RUN apt-get update -y && \
apt-get install -y --no-install-recommends wget xz-utils unzip && \ # Set up virtualenv
ENV VIRTUAL_ENV=/home/mediacms.io
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV PIP_NO_CACHE_DIR=1
RUN mkdir -p /home/mediacms.io/mediacms/{logs,pids} && cd /home/mediacms.io && python3 -m venv $VIRTUAL_ENV
# Install dependencies:
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . /home/mediacms.io/mediacms
WORKDIR /home/mediacms.io/mediacms
RUN wget -q http://zebulon.bok.net/Bento4/binaries/Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip && \
unzip Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip -d ../bento4 && \
mv ../bento4/Bento4-SDK-1-6-0-637.x86_64-unknown-linux/* ../bento4/ && \
rm -rf ../bento4/Bento4-SDK-1-6-0-637.x86_64-unknown-linux && \
rm -rf ../bento4/docs && \
rm Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip
############ RUNTIME IMAGE ############
FROM python:3.8-slim-buster as runtime-image
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV ADMIN_USER='admin'
ENV ADMIN_PASSWORD='mediacms'
ENV ADMIN_EMAIL='admin@localhost'
# See: https://github.com/celery/celery/issues/6285#issuecomment-715316219
ENV CELERY_APP='cms'
# Use these to toggle which processes supervisord should run
ENV ENABLE_UWSGI='yes'
ENV ENABLE_NGINX='yes'
ENV ENABLE_CELERY_BEAT='yes'
ENV ENABLE_CELERY_SHORT='yes'
ENV ENABLE_CELERY_LONG='yes'
ENV ENABLE_MIGRATIONS='yes'
# Set up virtualenv
ENV VIRTUAL_ENV=/home/mediacms.io
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
COPY --chown=www-data:www-data --from=compile-image /home/mediacms.io /home/mediacms.io
RUN apt-get update -y && apt-get -y upgrade && apt-get install --no-install-recommends \
supervisor nginx ffmpeg imagemagick procps -y && \
rm -rf /var/lib/apt/lists/* && \ rm -rf /var/lib/apt/lists/* && \
apt-get purge --auto-remove && \ apt-get purge --auto-remove && \
apt-get clean apt-get clean
RUN wget -q https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz
RUN mkdir -p ffmpeg-tmp && \
tar -xf ffmpeg-release-amd64-static.tar.xz --strip-components 1 -C ffmpeg-tmp && \
cp -v ffmpeg-tmp/ffmpeg ffmpeg-tmp/ffprobe ffmpeg-tmp/qt-faststart /usr/local/bin && \
rm -rf ffmpeg-tmp ffmpeg-release-amd64-static.tar.xz
# Install Bento4 in the specified location
RUN mkdir -p /home/mediacms.io/bento4 && \
wget -q http://zebulon.bok.net/Bento4/binaries/Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip && \
unzip Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip -d /home/mediacms.io/bento4 && \
mv /home/mediacms.io/bento4/Bento4-SDK-1-6-0-637.x86_64-unknown-linux/* /home/mediacms.io/bento4/ && \
rm -rf /home/mediacms.io/bento4/Bento4-SDK-1-6-0-637.x86_64-unknown-linux && \
rm -rf /home/mediacms.io/bento4/docs && \
rm Bento4-SDK-1-6-0-637.x86_64-unknown-linux.zip
############ BASE RUNTIME IMAGE ############
FROM python:3.13.5-slim-bookworm AS base
SHELL ["/bin/bash", "-c"]
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV CELERY_APP='cms'
ENV VIRTUAL_ENV=/home/mediacms.io
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
# Install system dependencies first
RUN apt-get update -y && \
apt-get -y upgrade && \
apt-get install --no-install-recommends -y \
supervisor \
nginx \
imagemagick \
procps \
build-essential \
pkg-config \
zlib1g-dev \
zlib1g \
libxml2-dev \
libxmlsec1-dev \
libxmlsec1-openssl \
libpq-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Set up virtualenv first
RUN mkdir -p /home/mediacms.io/mediacms/{logs} && \
cd /home/mediacms.io && \
python3 -m venv $VIRTUAL_ENV
# Copy requirements files
COPY requirements.txt requirements-dev.txt ./
# Install Python dependencies using pip (within virtualenv)
ARG DEVELOPMENT_MODE=False
RUN pip install --no-cache-dir uv && \
uv pip install --no-binary lxml --no-binary xmlsec -r requirements.txt && \
if [ "$DEVELOPMENT_MODE" = "True" ]; then \
echo "Installing development dependencies..." && \
uv pip install -r requirements-dev.txt; \
fi && \
apt-get purge -y --auto-remove \
build-essential \
pkg-config \
libxml2-dev \
libxmlsec1-dev \
libpq-dev
# Copy ffmpeg and Bento4 from build image
COPY --from=build-image /usr/local/bin/ffmpeg /usr/local/bin/ffmpeg
COPY --from=build-image /usr/local/bin/ffprobe /usr/local/bin/ffprobe
COPY --from=build-image /usr/local/bin/qt-faststart /usr/local/bin/qt-faststart
COPY --from=build-image /home/mediacms.io/bento4 /home/mediacms.io/bento4
# Copy application files
COPY . /home/mediacms.io/mediacms
WORKDIR /home/mediacms.io/mediacms WORKDIR /home/mediacms.io/mediacms
# required for sprite thumbnail generation for large video files
COPY deploy/docker/policy.xml /etc/ImageMagick-6/policy.xml
# Set process control environment variables
ENV ENABLE_UWSGI='yes' \
ENABLE_NGINX='yes' \
ENABLE_CELERY_BEAT='yes' \
ENABLE_CELERY_SHORT='yes' \
ENABLE_CELERY_LONG='yes' \
ENABLE_MIGRATIONS='yes'
EXPOSE 9000 80 EXPOSE 9000 80
RUN chmod +x ./deploy/docker/entrypoint.sh RUN chmod +x ./deploy/docker/entrypoint.sh
ENTRYPOINT ["./deploy/docker/entrypoint.sh"] ENTRYPOINT ["./deploy/docker/entrypoint.sh"]
CMD ["./deploy/docker/start.sh"]
############ FULL IMAGE ############ CMD ["./deploy/docker/start.sh"]
FROM base AS full
COPY requirements-full.txt ./
RUN mkdir -p /root/.cache/ && \
chmod go+rwx /root/ && \
chmod go+rwx /root/.cache/
RUN uv pip install -r requirements-full.txt

View File

@ -1,23 +0,0 @@
# History
## 3.0.0
### Features
- Updates Python/Django requirements and Dockerfile to use latest 3.11 Python - https://github.com/mediacms-io/mediacms/pull/826/files. This update requires some manual steps, for existing (not new) installations. Check the update section under the [Admin docs](https://github.com/mediacms-io/mediacms/blob/main/docs/admins_docs.md#2-server-installation), either for single server or for Docker Compose installations
- Upgrade postgres on Docker Compose - https://github.com/mediacms-io/mediacms/pull/749
### Fixes
- video player options for HLS - https://github.com/mediacms-io/mediacms/pull/832
- AVI videos not correctly recognised as videos - https://github.com/mediacms-io/mediacms/pull/833
## 2.1.0
### Fixes
- Increase uwsgi buffer-size parameter. This prevents an error by uwsgi with large headers - [#5b60](https://github.com/mediacms-io/mediacms/commit/5b601698a41ad97f08c1830e14b1c18f73ab8315)
- Fix issues with comments. These were not reported on the tracker but it is certain that they would not show comments on media files (non videos but also videos). Unfortunately this reverts work done with Timestamps on comments + Mentions on comments, more on PR [#802](https://github.com/mediacms-io/mediacms/pull/802)
### Features
- Allow tags to contains other characters too, not only English alphabet ones [#801](https://github.com/mediacms-io/mediacms/pull/801)
- Add simple cookie consent code [#799](https://github.com/mediacms-io/mediacms/pull/799)
- Allow password reset & email verify pages on global login required [#790](https://github.com/mediacms-io/mediacms/pull/790)
- Add api_url field to search api [#692](https://github.com/mediacms-io/mediacms/pull/692)

View File

@ -1,19 +0,0 @@
.PHONY: admin-shell build-frontend
admin-shell:
@container_id=$$(docker compose ps -q web); \
if [ -z "$$container_id" ]; then \
echo "Web container not found"; \
exit 1; \
else \
docker exec -it $$container_id /bin/bash; \
fi
build-frontend:
docker compose -f docker-compose-dev.yaml exec frontend npm run dist
cp -r frontend/dist/static/* static/
docker compose -f docker-compose-dev.yaml restart web
test:
docker compose -f docker-compose-dev.yaml exec --env TESTING=True -T web pytest

163
README.md
View File

@ -1,12 +1,6 @@
# MediaCMS ![MediaCMS](static/images/logo_dark.png)
[![GitHub license](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://raw.githubusercontent.com/mediacms-io/mediacms/main/LICENSE.txt) MediaCMS is a modern, fully featured open source video and media CMS. It is developed to meet the needs of modern web platforms for viewing and sharing media. It can be used to build a small to medium video and media portal within minutes.
[![Releases](https://img.shields.io/github/v/release/mediacms-io/mediacms?color=green)](https://github.com/mediacms-io/mediacms/releases/)
[![DockerHub](https://img.shields.io/docker/pulls/mediacms/mediacms)](https://hub.docker.com/r/mediacms/mediacms)
MediaCMS is a modern, fully featured open source video and media CMS. It is developed to meet the needs of modern web platforms for viewing and sharing media. It can be used to build a small to medium video and media portal within minutes.
It is built mostly using the modern stack Django + React and includes a REST API. It is built mostly using the modern stack Django + React and includes a REST API.
@ -23,15 +17,11 @@ A demo is available at https://demo.mediacms.io
## Features ## Features
- **Complete control over your data**: host it yourself! - **Complete control over your data**: host it yourself!
- **Modern technologies**: Django/Python/Celery, React.
- **Support for multiple publishing workflows**: public, private, unlisted and custom - **Support for multiple publishing workflows**: public, private, unlisted and custom
- **Role-Based Access Control (RBAC)**: create RBAC categories and connect users to groups with view/edit access on their media - **Modern technologies**: Django/Python/Celery, React.
- **Automatic transcription**: through integration with Whisper running locally
- **Multiple media types support**: video, audio, image, pdf - **Multiple media types support**: video, audio, image, pdf
- **Multiple media classification options**: categories, tags and custom - **Multiple media classification options**: categories, tags and custom
- **Multiple media sharing options**: social media share, videos embed code generation - **Multiple media sharing options**: social media share, videos embed code generation
- **Video Trimmer**: trim video, replace, save as new or create segments
- **SAML support**: with ability to add mappings to system roles and groups
- **Easy media searching**: enriched with live search functionality - **Easy media searching**: enriched with live search functionality
- **Playlists for audio and video content**: create playlists, add and reorder content - **Playlists for audio and video content**: create playlists, add and reorder content
- **Responsive design**: including light and dark themes - **Responsive design**: including light and dark themes
@ -39,83 +29,155 @@ A demo is available at https://demo.mediacms.io
- **Configurable actions**: allow download, add comments, add likes, dislikes, report media - **Configurable actions**: allow download, add comments, add likes, dislikes, report media
- **Configuration options**: change logos, fonts, styling, add more pages - **Configuration options**: change logos, fonts, styling, add more pages
- **Enhanced video player**: customized video.js player with multiple resolution and playback speed options - **Enhanced video player**: customized video.js player with multiple resolution and playback speed options
- **Multiple transcoding profiles**: sane defaults for multiple dimensions (144p, 240p, 360p, 480p, 720p, 1080p) and multiple profiles (h264, h265, vp9) - **Multiple transcoding profiles**: sane defaults for multiple dimensions (240p, 360p, 480p, 720p, 1080p) and multiple profiles (h264, h265, vp9)
- **Adaptive video streaming**: possible through HLS protocol - **Adaptive video streaming**: possible through HLS protocol
- **Subtitles/CC**: support for multilingual subtitle files - **Subtitles/CC**: support for multilingual subtitle files
- **Scalable transcoding**: transcoding through priorities. Experimental support for remote workers - **Scalable transcoding**: transcoding through priorities. Experimental support for remote workers
- **Chunked file uploads**: for pausable/resumable upload of content - **Chunked file uploads**: for pausable/resumable upload of content
- **REST API**: Documented through Swagger
- **Translation**: Most of the CMS is translated to a number of languages
## Example cases ## Example cases
- **Universities, schools, education.** Administrators and editors keep what content will be published, students are not distracted with advertisements and irrelevant content, plus they have the ability to select either to stream or download content. - **Schools, education.** Administrators and editors keep what content will be published, students are not distracted with advertisements and irrelevant content, plus they have the ability to select either to stream or download content.
- **Organization sensitive content.** In cases where content is sensitive and cannot be uploaded to external sites. - **Organization sensitive content.** In cases where content is sensitive and cannot be uploaded to external sites.
- **Build a great community.** MediaCMS can be customized (URLs, logos, fonts, aesthetics) so that you create a highly customized video portal for your community! - **Build a great community.** MediaCMS can be customized (URLs, logos, fonts, aesthetics) so that you create a highly customized video portal for your community!
- **Personal portal.** Organize, categorize and host your content the way you prefer. - **Personal portal.** Organize, categorize and host your content the way you prefer.
## Philosophy ## Philosophy
We believe there's a need for quality open source web applications that can be used to build community portals and support collaboration. We believe there's a need for quality open source web applications that can be used to build community portals and support collaboration.
We have three goals for MediaCMS: a) deliver all functionality one would expect from a modern system, b) allow for easy installation and maintenance, c) allow easy customization and addition of features.
We have three goals for MediaCMS: a) deliver all functionality one would expect from a modern system, b) allow for easy installation and maintenance, c) allow easy customization and addition of features.
## License ## License
MediaCMS is released under [GNU Affero General Public License v3.0 license](LICENSE.txt). MediaCMS is released under [GNU Affero General Public License v3.0 license](LICENSE.txt).
Copyright Markos Gogoulos. Copyright Markos Gogoulos and Yiannis Stergiou
## Support and paid services ## Support and paid services
We provide custom installations, development of extra functionality, migration from existing systems, integrations with legacy systems, training and support. Contact us at info@mediacms.io for more information. We provide custom installations, development of extra functionality, migration from existing systems, integrations with legacy systems, training and support. Contact us at info@mediacms.io for more information.
### Commercial Hostings
**Elestio**
You can deploy MediaCMS on Elestio using one-click deployment. Elestio supports MediaCMS by providing revenue share so go ahead and click below to deploy and use MediaCMS.
[![Deploy on Elestio](https://elest.io/images/logos/deploy-to-elestio-btn.png)](https://elest.io/open-source/mediacms) ## Hardware dependencies
## Hardware considerations For a small to medium installation, with a few hours of video uploaded daily, and a few hundreds of active daily users viewing content, 4GB Ram / 2-4 CPUs as minimum is ok. For a larger installation with many hours of video uploaded daily, consider adding more CPUs and more Ram.
For a small to medium installation, with a few hours of video uploaded daily, and a few hundreds of active daily users viewing content, 4GB Ram / 2-4 CPUs as minimum is ok. For a larger installation with many hours of video uploaded daily, consider adding more CPUs and more Ram.
In terms of disk space, think of what the needs will be. A general rule is to multiply by three the size of the expected uploaded videos (since the system keeps original versions, encoded versions plus HLS), so if you receive 1G of videos daily and maintain all of them, you should consider a 1T disk across a year (1G * 3 * 365). In terms of disk space, think of what the needs will be. A general rule is to multiply by three the size of the expected uploaded videos (since the system keeps original versions, encoded versions plus HLS), so if you receive 1G of videos daily and maintain all of them, you should consider a 1T disk across a year (1G * 3 * 365).
In order to support automatic transcriptions through Whisper, consider more CPUs.
## Installation / Maintanance ## Installation
There are two ways to run MediaCMS, through Docker Compose and through installing it on a server via an automation script that installs and configures all needed services. Find the related pages: There are two ways to run MediaCMS, through Docker Compose and through installing it on a server via an automation script that installs and configures all needed services.
- [Single Server](docs/admins_docs.md#2-server-installation) page ### Docker Compose installation
- [Docker Compose](docs/admins_docs.md#3-docker-installation) page Install a recent version of [Docker](https://docs.docker.com/get-docker/), and [Docker Compose](https://docs.docker.com/compose/install/).
A complete guide can be found on the blog post [How to self-host and share your videos in 2021](https://medium.com/@MediaCMS.io/how-to-self-host-and-share-your-videos-in-2021-14067e3b291b). Run as root
## Documentation ```bash
git clone https://github.com/mediacms-io/mediacms
cd mediacms
```
* [Users documentation](docs/user_docs.md) page The default option is to serve MediaCMS on all ips available of the server (including localhost).
* [Administrators documentation](docs/admins_docs.md) page
* [Developers documentation](docs/developers_docs.md) page Now run
* [Configuration](docs/admins_docs.md#5-configuration) page
* [Transcoding](docs/transcoding.md) page ```bash
* [Developer Experience](docs/dev_exp.md) page docker-compose up
* [Media Permissions](docs/media_permissions.md) page ```
This will download all MediaCMS related Docker images and start all containers. Once it finishes, MediaCMS will be installed and available on http://localhost or http://ip
For more instructions, checkout the docs on the [Docker deployment](docs/Docker_deployment.md) page. Docker Compose support has been contributed by @swiftugandan.
### Single server installation
The core dependencies are Python3, Django3, Celery, PostgreSQL, Redis, ffmpeg. Any system that can have these dependencies installed, can run MediaCMS. But we strongly suggest installing on Linux Ubuntu 18 or 20 versions.
Installation on a Ubuntu 18 or 20 system with git utility installed should be completed in a few minutes with the following steps.
Make sure you run it as user root, on a clear system, since the automatic script will install and configure the following services: Celery/PostgreSQL/Redis/Nginx and will override any existing settings.
Automated script - to run on Ubuntu 18 or Ubuntu 20 flavors only!
```bash
mkdir /home/mediacms.io && cd /home/mediacms.io/
git clone https://github.com/mediacms-io/mediacms
cd /home/mediacms.io/mediacms/ && bash ./install.sh
```
The script will ask if you have a URL where you want to deploy MediaCMS, otherwise it will use localhost. If you provide a URL, it will use Let's Encrypt service to install a valid ssl certificate.
## Update
If you've used the above way to install MediaCMS, update with the following:
```bash
cd /home/mediacms.io/mediacms # enter mediacms directory
source /home/mediacms.io/bin/activate # use virtualenv
git pull # update code
python manage.py migrate # run Django migrations
sudo systemctl restart mediacms celery_long celery_short # restart services
```
## Configure
Several options are available on cms/settings.py, most of the things that are allowed or should be disallowed are described there. It is advisable to override any of them by adding it to cms/local_settings.py. All configuration options will be documented gradually on the [Configuration](docs/Configuration.md) page.
## Authors
MediaCMS is developed by Yiannis Stergiou and Markos Gogoulos. We are Wordgames - https://wordgames.gr.
## Technology ## Technology
This software uses the following list of awesome technologies:
This software uses the following list of awesome technologies: Python, Django, Django Rest Framework, Celery, PostgreSQL, Redis, Nginx, uWSGI, React, Fine Uploader, video.js, FFMPEG, Bento4 - Python
- Django
- Django Rest Framework
- Celery
- PostgreSQL
- Redis
- Nginx
- uWSGI
- React
- Fine Uploader
- video.js
- FFMPEG
- Bento4
## Who is using it ## Who is using it
- **Multiple Universities** for hosting educational videos
- **Cinemata** non-profit media, technology and culture organization - https://cinemata.org - **EngageMedia** non-profit media, technology and culture organization - https://video.engagemedia.org
- **Critical Commons** public media archive and fair use advocacy network - https://criticalcommons.org - **Critical Commons** public media archive and fair use advocacy network - https://criticalcommons.org
- **American Association of Gynecologic Laparoscopists** - https://surgeryu.org/
- **Heritales** International Heritage Film Festival - https://stage.heritales.org
## Thanks To
- **Anna Helme**, for such a great partnership all these years!
- **Steve Anderson**, for trusting us and helping the Wordgames team make this real.
- **Andrew Lowenthal, King Catoy, Rezwan Islam** and the rest of the great team of [Engage Media](https://engagemedia.org).
- **Ioannis Korovesis, Ioannis Maistros, Diomidis Spinellis and Theodoros Karounos**, for their mentorship all these years, their contribution to science and the promotion of open source and free software technologies.
- **Antonis Ikonomou**, for hosting us on the excellent [Innovathens](https://www.innovathens.gr) space.
- **Werner Robitza**, for helping us with ffmpeg related stuff.
## How to contribute ## How to contribute
@ -125,12 +187,9 @@ If you like the project, here's a few things you can do
- Suggest us to others that are interested to hire us - Suggest us to others that are interested to hire us
- Write a blog post/article about MediaCMS - Write a blog post/article about MediaCMS
- Share on social media about the project - Share on social media about the project
- Open issues, participate on [discussions](https://github.com/mediacms-io/mediacms/discussions), report bugs, suggest ideas - Open issues, participate on discussions, report bugs, suggest ideas
- [Show and tell](https://github.com/mediacms-io/mediacms/discussions/categories/show-and-tell) how you are using the project
- Star the project - Star the project
- Add functionality, work on a PR, fix an issue! - Add functionality, work on a PR, fix an issue!
## Contact ## Contact
info@mediacms.io info@mediacms.io

View File

@ -4,6 +4,7 @@ from django.db import migrations, models
class Migration(migrations.Migration): class Migration(migrations.Migration):
initial = True initial = True
dependencies = [] dependencies = []

View File

@ -1,10 +1,11 @@
# Generated by Django 3.1.4 on 2020-12-01 07:12 # Generated by Django 3.1.4 on 2020-12-01 07:12
import django.db.models.deletion
from django.db import migrations, models from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration): class Migration(migrations.Migration):
initial = True initial = True
dependencies = [ dependencies = [

View File

@ -1,11 +1,12 @@
# Generated by Django 3.1.4 on 2020-12-01 07:12 # Generated by Django 3.1.4 on 2020-12-01 07:12
import django.db.models.deletion
from django.conf import settings from django.conf import settings
from django.db import migrations, models from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration): class Migration(migrations.Migration):
initial = True initial = True
dependencies = [ dependencies = [
@ -34,6 +35,8 @@ class Migration(migrations.Migration):
), ),
migrations.AddIndex( migrations.AddIndex(
model_name="mediaaction", model_name="mediaaction",
index=models.Index(fields=["session_key", "action"], name="actions_med_session_fac55a_idx"), index=models.Index(
fields=["session_key", "action"], name="actions_med_session_fac55a_idx"
),
), ),
] ]

View File

@ -1,7 +1,6 @@
from django.db import models from django.db import models
from files.models import Media
from users.models import User from users.models import User
from files.models import Media
USER_MEDIA_ACTIONS = ( USER_MEDIA_ACTIONS = (
("like", "Like"), ("like", "Like"),
@ -31,11 +30,15 @@ class MediaAction(models.Model):
help_text="for not logged in users", help_text="for not logged in users",
) )
action = models.CharField(max_length=20, choices=USER_MEDIA_ACTIONS, default="watch") action = models.CharField(
max_length=20, choices=USER_MEDIA_ACTIONS, default="watch"
)
# keeps extra info, eg on report action, why it is reported # keeps extra info, eg on report action, why it is reported
extra_info = models.TextField(blank=True, null=True) extra_info = models.TextField(blank=True, null=True)
media = models.ForeignKey(Media, on_delete=models.CASCADE, related_name="mediaactions") media = models.ForeignKey(
Media, on_delete=models.CASCADE, related_name="mediaactions"
)
action_date = models.DateTimeField(auto_now_add=True) action_date = models.DateTimeField(auto_now_add=True)
remote_ip = models.CharField(max_length=40, blank=True, null=True) remote_ip = models.CharField(max_length=40, blank=True, null=True)

View File

@ -1,86 +0,0 @@
from django.apps import AppConfig
from django.conf import settings
from django.contrib import admin
class AdminCustomizationsConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'admin_customizations'
def ready(self):
original_get_app_list = admin.AdminSite.get_app_list
def get_app_list(self, request, app_label=None):
"""Custom get_app_list"""
app_list = original_get_app_list(self, request, app_label)
# To see the list:
# print([a.get('app_label') for a in app_list])
email_model = None
rbac_group_model = None
identity_providers_user_log_model = None
identity_providers_login_option = None
auth_app = None
rbac_app = None
socialaccount_app = None
for app in app_list:
if app['app_label'] == 'users':
auth_app = app
elif app['app_label'] == 'account':
for model in app['models']:
if model['object_name'] == 'EmailAddress':
email_model = model
elif app['app_label'] == 'rbac':
if not getattr(settings, 'USE_RBAC', False):
continue
rbac_app = app
for model in app['models']:
if model['object_name'] == 'RBACGroup':
rbac_group_model = model
elif app['app_label'] == 'identity_providers':
if not getattr(settings, 'USE_IDENTITY_PROVIDERS', False):
continue
models_to_check = list(app['models'])
for model in models_to_check:
if model['object_name'] == 'IdentityProviderUserLog':
identity_providers_user_log_model = model
if model['object_name'] == 'LoginOption':
identity_providers_login_option = model
elif app['app_label'] == 'socialaccount':
socialaccount_app = app
if email_model and auth_app:
auth_app['models'].append(email_model)
if rbac_group_model and rbac_app and auth_app:
auth_app['models'].append(rbac_group_model)
if identity_providers_login_option and socialaccount_app:
socialaccount_app['models'].append(identity_providers_login_option)
if identity_providers_user_log_model and socialaccount_app:
socialaccount_app['models'].append(identity_providers_user_log_model)
# 2. don't include the following apps
apps_to_hide = ['authtoken', 'auth', 'account', 'saml_auth', 'rbac']
if not getattr(settings, 'USE_RBAC', False):
apps_to_hide.append('rbac')
if not getattr(settings, 'USE_IDENTITY_PROVIDERS', False):
apps_to_hide.append('socialaccount')
app_list = [app for app in app_list if app['app_label'] not in apps_to_hide]
# 3. change the ordering
app_order = {
'files': 1,
'users': 2,
'socialaccount': 3,
'rbac': 5,
}
app_list.sort(key=lambda x: app_order.get(x['app_label'], 999))
return app_list
admin.AdminSite.get_app_list = get_app_list

View File

@ -1,10 +0,0 @@
## MediaCMS CLI Tool
This is the CLI tool to interact with the API of your installation/instance of MediaCMS.
### How to configure and use the tools
- Make sure that you have all the required installations (`cli-tool/requirements.txt`)installed. To install it -
- Create a new virtualenv using any python virtualenv manager.
- Then activate the virtualenv and enter `pip install -r requirements.txt`.
- Create an .env file in this folder (`mediacms/cli-tool/`)
- Run the cli tool using the command `python cli.py login`. This will authenticate you and store necessary creds for further authentications.
- To check the credentials and necessary setup, run `python cli.py whoami`. This will show your details.

View File

@ -1,167 +0,0 @@
import json
import os
import click
import requests
from decouple import config
from rich import print
from rich.console import Console
from rich.table import Table
console = Console()
print("Welcome to the CLI Tool of [bold blue]MediaCMS![/bold blue]", ":thumbs_up:")
BASE_URL = 'https://demo.mediacms.io/api/v1'
AUTH_KEY = ''
USERNAME = ''
EMAIL = ''
def set_envs():
with open('.env', 'r') as file:
if not file.read(1):
print("Use the Login command to set your credential environment variables")
else:
global AUTH_KEY, USERNAME, EMAIL
AUTH_KEY = config('AUTH_KEY')
USERNAME = config('USERNAME')
EMAIL = config('EMAIL')
set_envs()
@click.group()
def apis():
"""A CLI wrapper for the MediaCMS API endpoints."""
@apis.command()
def login():
"""Login to your account."""
email = input('Enter your email address: ')
password = input('Enter your password: ')
data = {
"email": f"{email}",
"password": f"{password}",
}
response = requests.post(url=f'{BASE_URL}/login', data=data)
if response.status_code == 200:
username = json.loads(response.text)["username"]
with open(".env", "w") as file:
file.writelines(f'AUTH_KEY={json.loads(response.text)["token"]}\n')
file.writelines(f'EMAIL={json.loads(response.text)["email"]}\n')
file.writelines(f'USERNAME={json.loads(response.text)["username"]}\n')
print(f"Welcome to MediaCMS [bold blue]{username}[/bold blue]. Your auth creds have been suceesfully stored in the .env file", ":v:")
else:
print(f'Error: {"non_field_errors": ["User not found."]}')
@apis.command()
def upload_media():
"""Upload media to the server"""
headers = {'authorization': f'Token {AUTH_KEY}'}
path = input('Enter the location of the file or directory where multiple files are present: ')
if os.path.isdir(path):
for filename in os.listdir(path):
files = {}
abs = os.path.abspath(f"{path}/{filename}")
files['media_file'] = open(f'{abs}', 'rb')
response = requests.post(url=f'{BASE_URL}/media', headers=headers, files=files)
if response.status_code == 201:
print(f"[bold blue]{filename}[/bold blue] successfully uploaded!")
else:
print(f'Error: {response.text}')
else:
files = {}
files['media_file'] = open(f'{os.path.abspath(path)}', 'rb')
response = requests.post(url=f'{BASE_URL}/media', headers=headers, files=files)
if response.status_code == 201:
print(f"[bold blue]{filename}[/bold blue] successfully uploaded!")
else:
print(f'Error: {response.text}')
@apis.command()
def my_media():
"""List all my media"""
headers = {'authorization': f'Token {AUTH_KEY}'}
response = requests.get(url=f'{BASE_URL}/media?author={USERNAME}', headers=headers)
if response.status_code == 200:
data_json = json.loads(response.text)
table = Table(show_header=True, header_style="bold magenta")
table.add_column("Name of the media")
table.add_column("Media Type")
table.add_column("State")
for data in data_json['results']:
table.add_row(data['title'], data['media_type'], data['state'])
console.print(table)
else:
print(f'Could not get the media: {response.text}')
@apis.command()
def whoami():
"""Shows the details of the authorized user"""
headers = {'authorization': f'Token {AUTH_KEY}'}
response = requests.get(url=f'{BASE_URL}/whoami', headers=headers)
for data, value in json.loads(response.text).items():
print(data, ' : ', value)
@apis.command()
def categories():
"""List all categories."""
response = requests.get(url=f'{BASE_URL}/categories')
if response.status_code == 200:
data_json = json.loads(response.text)
table = Table(show_header=True, header_style="bold magenta")
table.add_column("Category")
table.add_column("Description")
for data in data_json:
table.add_row(data['title'], data['description'])
console.print(table)
else:
print(f'Could not get the categories: {response.text}')
@apis.command()
def encodings():
"""List all encoding profiles"""
response = requests.get(url=f'{BASE_URL}/encode_profiles/')
if response.status_code == 200:
data_json = json.loads(response.text)
table = Table(show_header=True, header_style="bold magenta")
table.add_column("Name")
table.add_column("Extension")
table.add_column("Resolution")
table.add_column("Codec")
table.add_column("Description")
for data in data_json:
table.add_row(data['name'], data['extension'], str(data['resolution']), data['codec'], data['description'])
console.print(table)
else:
print(f'Could not get the encodings: {response.text}')
if __name__ == '__main__':
apis()

View File

@ -1,4 +0,0 @@
click
python-decouple
requests
rich

View File

@ -1,5 +1,4 @@
from __future__ import absolute_import from __future__ import absolute_import
from .celery import app as celery_app from .celery import app as celery_app
__all__ = ["celery_app"] __all__ = ["celery_app"]

View File

@ -1,10 +0,0 @@
from django.conf import settings
from django.contrib.auth.backends import ModelBackend
class ApprovalBackend(ModelBackend):
def user_can_authenticate(self, user):
can_authenticate = super().user_can_authenticate(user)
if can_authenticate and settings.USERS_NEEDS_TO_BE_APPROVED and not user.is_superuser:
return getattr(user, 'is_approved', False)
return can_authenticate

View File

@ -1,9 +1,6 @@
from __future__ import absolute_import from __future__ import absolute_import
import os import os
from celery import Celery from celery import Celery
from django.conf import settings
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cms.settings") os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cms.settings")
app = Celery("cms") app = Celery("cms")
@ -15,8 +12,5 @@ app.conf.beat_schedule = app.conf.CELERY_BEAT_SCHEDULE
app.conf.broker_transport_options = {"visibility_timeout": 60 * 60 * 24} # 1 day app.conf.broker_transport_options = {"visibility_timeout": 60 * 60 * 24} # 1 day
# http://docs.celeryproject.org/en/latest/getting-started/brokers/redis.html#redis-caveats # http://docs.celeryproject.org/en/latest/getting-started/brokers/redis.html#redis-caveats
# setting this to settings.py file only is not respected. Setting here too
app.conf.task_always_eager = settings.CELERY_TASK_ALWAYS_EAGER
app.conf.worker_prefetch_multiplier = 1 app.conf.worker_prefetch_multiplier = 1

View File

@ -1,9 +1,8 @@
from collections import OrderedDict # requires Python 2.7 or later
from django.core.paginator import Paginator
from django.utils.functional import cached_property
from rest_framework.pagination import PageNumberPagination from rest_framework.pagination import PageNumberPagination
from rest_framework.response import Response from rest_framework.response import Response
from collections import OrderedDict # requires Python 2.7 or later
from django.core.paginator import Paginator
from django.utils.functional import cached_property
class FasterDjangoPaginator(Paginator): class FasterDjangoPaginator(Paginator):
@ -18,6 +17,7 @@ class FastPaginationWithoutCount(PageNumberPagination):
django_paginator_class = FasterDjangoPaginator django_paginator_class = FasterDjangoPaginator
def get_paginated_response(self, data): def get_paginated_response(self, data):
return Response( return Response(
OrderedDict( OrderedDict(
[ [

View File

@ -1,57 +0,0 @@
# Development settings, used in docker-compose-dev.yaml
import os
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
INSTALLED_APPS = [
"admin_customizations",
"django.contrib.auth",
"allauth",
"allauth.account",
"allauth.socialaccount",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
"jazzmin",
"django.contrib.admin",
"django.contrib.sites",
"rest_framework",
"rest_framework.authtoken",
"imagekit",
"files.apps.FilesConfig",
"users.apps.UsersConfig",
"actions.apps.ActionsConfig",
"rbac.apps.RbacConfig",
"identity_providers.apps.IdentityProvidersConfig",
"debug_toolbar",
"mptt",
"crispy_forms",
"crispy_bootstrap5",
"uploader.apps.UploaderConfig",
"djcelery_email",
"drf_yasg",
"allauth.socialaccount.providers.saml",
"saml_auth.apps.SamlAuthConfig",
"corsheaders",
"tinymce",
]
MIDDLEWARE = [
'corsheaders.middleware.CorsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
"django.middleware.locale.LocaleMiddleware",
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'debug_toolbar.middleware.DebugToolbarMiddleware',
"allauth.account.middleware.AccountMiddleware",
]
DEBUG = True
CORS_ORIGIN_ALLOW_ALL = True
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
STATIC_ROOT = os.path.join(BASE_DIR, 'static_collected')

View File

@ -1,23 +0,0 @@
from django.conf import settings
from django.http import JsonResponse
from django.shortcuts import redirect
from django.urls import reverse
class ApprovalMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
if settings.USERS_NEEDS_TO_BE_APPROVED and request.user.is_authenticated and not request.user.is_superuser and not getattr(request.user, 'is_approved', False):
allowed_paths = [
reverse('approval_required'),
reverse('account_logout'),
]
if request.path not in allowed_paths:
if request.path.startswith('/api/'):
return JsonResponse({'detail': 'User account not approved.'}, status=403)
return redirect('approval_required')
response = self.get_response(request)
return response

View File

@ -1,29 +1,13 @@
from django.conf import settings from django.conf import settings
from rest_framework import permissions from rest_framework import permissions
from rest_framework.exceptions import PermissionDenied from files.methods import is_mediacms_editor, is_mediacms_manager
from files.methods import (
is_mediacms_editor,
is_mediacms_manager,
user_allowed_to_upload,
)
class IsAuthorizedToAdd(permissions.BasePermission): class IsAuthorizedToAdd(permissions.BasePermission):
def has_permission(self, request, view): def has_permission(self, request, view):
if request.method in permissions.SAFE_METHODS: if request.method in permissions.SAFE_METHODS:
return True return True
if not user_allowed_to_upload(request): return user_allowed_to_upload(request)
raise PermissionDenied("You don't have permission to upload media, or have reached max number of media uploads.")
return True
class IsAuthorizedToAddComment(permissions.BasePermission):
def has_permission(self, request, view):
if request.method in permissions.SAFE_METHODS:
return True
return user_allowed_to_comment(request)
class IsUserOrManager(permissions.BasePermission): class IsUserOrManager(permissions.BasePermission):
@ -40,10 +24,7 @@ class IsUserOrManager(permissions.BasePermission):
if is_mediacms_manager(request.user): if is_mediacms_manager(request.user):
return True return True
if hasattr(obj, 'user'): return obj.user == request.user
return obj.user == request.user
else:
return obj == request.user
class IsUserOrEditor(permissions.BasePermission): class IsUserOrEditor(permissions.BasePermission):
@ -63,22 +44,21 @@ class IsUserOrEditor(permissions.BasePermission):
return obj.user == request.user return obj.user == request.user
def user_allowed_to_comment(request): def user_allowed_to_upload(request):
"""Any custom logic for whether a user is allowed """Any custom logic for whether a user is allowed
to comment lives here to upload content lives here
""" """
if request.user.is_anonymous: if request.user.is_anonymous:
return False return False
if request.user.is_superuser: if request.user.is_superuser:
return True return True
# Default is "all" if settings.CAN_ADD_MEDIA == "all":
if not hasattr(settings, "CAN_COMMENT") or settings.CAN_COMMENT == "all":
return True return True
elif settings.CAN_COMMENT == "email_verified": elif settings.CAN_ADD_MEDIA == "email_verified":
if request.user.email_is_verified: if request.user.email_is_verified:
return True return True
elif settings.CAN_COMMENT == "advancedUser": elif settings.CAN_ADD_MEDIA == "advancedUser":
if request.user.advancedUser: if request.user.advancedUser:
return True return True
return False return False

View File

@ -1,29 +1,22 @@
import os import os
from celery.schedules import crontab from celery.schedules import crontab
from django.utils.translation import gettext_lazy as _
DEBUG = False DEBUG = False
# PORTAL NAME, this is the portal title and # PORTAL NAME, this is the portal title and
# is also shown on several places as emails # is also shown on several places as emails
PORTAL_NAME = "MediaCMS" PORTAL_NAME = "MediaCMS"
PORTAL_DESCRIPTION = "" LANGUAGE_CODE = "en-us"
TIME_ZONE = "Europe/London" TIME_ZONE = "Europe/London"
# who can add media # who can add media
# valid options include 'all', 'email_verified', 'advancedUser' # valid options include 'all', 'email_verified', 'advancedUser'
CAN_ADD_MEDIA = "all" CAN_ADD_MEDIA = "all"
# who can comment
# valid options include 'all', 'email_verified', 'advancedUser'
CAN_COMMENT = "all"
# valid choices here are 'public', 'private', 'unlisted # valid choices here are 'public', 'private', 'unlisted
PORTAL_WORKFLOW = "public" PORTAL_WORKFLOW = "public"
# valid values: 'light', 'dark'. DEFAULT_THEME = "black" # this is not taken under consideration currently
DEFAULT_THEME = "light"
# These are passed on every request # These are passed on every request
@ -91,48 +84,26 @@ MAX_MEDIA_PER_PLAYLIST = 70
UPLOAD_MAX_SIZE = 800 * 1024 * 1000 * 5 UPLOAD_MAX_SIZE = 800 * 1024 * 1000 * 5
MAX_CHARS_FOR_COMMENT = 10000 # so that it doesn't end up huge MAX_CHARS_FOR_COMMENT = 10000 # so that it doesn't end up huge
TIMESTAMP_IN_TIMEBAR = False # shows timestamped comments in the timebar for videos
ALLOW_MENTION_IN_COMMENTS = False # allowing to mention other users with @ in the comments
# valid options: content, author # valid options: content, author
RELATED_MEDIA_STRATEGY = "content" RELATED_MEDIA_STRATEGY = "content"
# Whether or not to generate a sitemap.xml listing the pages on the site (default: False)
GENERATE_SITEMAP = False
USE_I18N = True USE_I18N = True
USE_L10N = True USE_L10N = True
USE_TZ = True USE_TZ = True
SITE_ID = 1 SITE_ID = 1
# these are the portal logos (dark and light)
# set new paths for svg or png if you want to override
# svg has priority over png, so if you want to use
# custom pngs and not svgs, remove the lines with svgs
# or set as empty strings
# example:
# PORTAL_LOGO_DARK_SVG = ""
# PORTAL_LOGO_LIGHT_SVG = ""
# place the files on static/images folder
PORTAL_LOGO_DARK_SVG = "/static/images/logo_dark.svg"
PORTAL_LOGO_DARK_PNG = "/static/images/logo_dark.png"
PORTAL_LOGO_LIGHT_SVG = "/static/images/logo_light.svg"
PORTAL_LOGO_LIGHT_PNG = "/static/images/logo_dark.png"
# paths to extra css files to be included, eg "/static/css/custom.css"
# place css inside static/css folder
EXTRA_CSS_PATHS = []
# protection agains anonymous users # protection agains anonymous users
# per ip address limit, for actions as like/dislike/report # per ip address limit, for actions as like/dislike/report
TIME_TO_ACTION_ANONYMOUS = 10 * 60 TIME_TO_ACTION_ANONYMOUS = 10 * 60
# django-allauth settings # django-allauth settings
ACCOUNT_SESSION_REMEMBER = True ACCOUNT_SESSION_REMEMBER = True
ACCOUNT_LOGIN_METHODS = {"username", "email"} ACCOUNT_AUTHENTICATION_METHOD = "username_email"
ACCOUNT_EMAIL_REQUIRED = True # new users need to specify email ACCOUNT_EMAIL_REQUIRED = True # new users need to specify email
ACCOUNT_EMAIL_VERIFICATION = "optional" # 'mandatory' 'none' ACCOUNT_EMAIL_VERIFICATION = "optional" # 'mandatory' 'none'
ACCOUNT_LOGIN_ON_EMAIL_CONFIRMATION = True ACCOUNT_LOGIN_ON_EMAIL_CONFIRMATION = True
ACCOUNT_USERNAME_MIN_LENGTH = 4 ACCOUNT_USERNAME_MIN_LENGTH = "4"
ACCOUNT_ADAPTER = "users.adapter.MyAccountAdapter" ACCOUNT_ADAPTER = "users.adapter.MyAccountAdapter"
ACCOUNT_SIGNUP_FORM_CLASS = "users.forms.SignupForm" ACCOUNT_SIGNUP_FORM_CLASS = "users.forms.SignupForm"
ACCOUNT_USERNAME_VALIDATORS = "users.validators.custom_username_validators" ACCOUNT_USERNAME_VALIDATORS = "users.validators.custom_username_validators"
@ -140,19 +111,13 @@ ACCOUNT_SIGNUP_PASSWORD_ENTER_TWICE = False
ACCOUNT_USERNAME_REQUIRED = True ACCOUNT_USERNAME_REQUIRED = True
ACCOUNT_LOGIN_ON_PASSWORD_RESET = True ACCOUNT_LOGIN_ON_PASSWORD_RESET = True
ACCOUNT_EMAIL_CONFIRMATION_EXPIRE_DAYS = 1 ACCOUNT_EMAIL_CONFIRMATION_EXPIRE_DAYS = 1
ACCOUNT_LOGIN_ATTEMPTS_LIMIT = 20
ACCOUNT_LOGIN_ATTEMPTS_TIMEOUT = 5
# registration won't be open, might also consider to remove links for register # registration won't be open, might also consider to remove links for register
USERS_CAN_SELF_REGISTER = True USERS_CAN_SELF_REGISTER = True
RESTRICTED_DOMAINS_FOR_USER_REGISTRATION = ["xxx.com", "emaildomainwhatever.com"] RESTRICTED_DOMAINS_FOR_USER_REGISTRATION = ["xxx.com", "emaildomainwhatever.com"]
# by default users do not need to be approved. If this is set to True, then new users
# will have to be approved before they can login successfully
USERS_NEEDS_TO_BE_APPROVED = False
# Comma separated list of domains: ["organization.com", "private.organization.com", "org2.com"]
# Empty list disables.
ALLOWED_DOMAINS_FOR_USER_REGISTRATION = []
# django rest settings # django rest settings
REST_FRAMEWORK = { REST_FRAMEWORK = {
"DEFAULT_AUTHENTICATION_CLASSES": ( "DEFAULT_AUTHENTICATION_CLASSES": (
@ -179,13 +144,10 @@ STATIC_ROOT = BASE_DIR + "/static/"
# where uploaded + encoded media are stored # where uploaded + encoded media are stored
MEDIA_ROOT = BASE_DIR + "/media_files/" MEDIA_ROOT = BASE_DIR + "/media_files/"
# these used to be os.path.join(MEDIA_ROOT, "folder/") but update to MEDIA_UPLOAD_DIR = os.path.join(MEDIA_ROOT, "original/")
# Django 3.1.9 requires not absolute paths to be utilized... MEDIA_ENCODING_DIR = os.path.join(MEDIA_ROOT, "encoded/")
THUMBNAIL_UPLOAD_DIR = os.path.join(MEDIA_UPLOAD_DIR, "thumbnails/")
MEDIA_UPLOAD_DIR = "original/" SUBTITLES_UPLOAD_DIR = os.path.join(MEDIA_UPLOAD_DIR, "subtitles/")
MEDIA_ENCODING_DIR = "encoded/"
THUMBNAIL_UPLOAD_DIR = f"{MEDIA_UPLOAD_DIR}/thumbnails/"
SUBTITLES_UPLOAD_DIR = f"{MEDIA_UPLOAD_DIR}/subtitles/"
HLS_DIR = os.path.join(MEDIA_ROOT, "hls/") HLS_DIR = os.path.join(MEDIA_ROOT, "hls/")
FFMPEG_COMMAND = "ffmpeg" # this is the path FFMPEG_COMMAND = "ffmpeg" # this is the path
@ -207,7 +169,7 @@ CHUNKIZE_VIDEO_DURATION = 60 * 5
VIDEO_CHUNKS_DURATION = 60 * 4 VIDEO_CHUNKS_DURATION = 60 * 4
# always get these two, even if upscaling # always get these two, even if upscaling
MINIMUM_RESOLUTIONS_TO_ENCODE = [144, 240] MINIMUM_RESOLUTIONS_TO_ENCODE = [240, 360]
# default settings for notifications # default settings for notifications
# not all of them are implemented # not all of them are implemented
@ -247,13 +209,15 @@ POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY = ""
# only in case where unlisted workflow is used and no commentary # only in case where unlisted workflow is used and no commentary
# exists # exists
CANNOT_ADD_MEDIA_MESSAGE = "User cannot add media, or maximum number of media uploads has been reached." CANNOT_ADD_MEDIA_MESSAGE = ""
# mp4hls command, part of Bento4 # mp4hls command, part of Bendo4
MP4HLS_COMMAND = "/home/mediacms.io/mediacms/Bento4-SDK-1-6-0-637.x86_64-unknown-linux/bin/mp4hls" MP4HLS_COMMAND = (
"/home/mediacms.io/mediacms/Bento4-SDK-1-6-0-637.x86_64-unknown-linux/bin/mp4hls"
)
# highly experimental, related with remote workers # highly experimental, related with remote workers
ADMIN_TOKEN = "" ADMIN_TOKEN = "c2b8e1838b6128asd333ddc5e24"
# this is used by remote workers to push # this is used by remote workers to push
# encodings once they are done # encodings once they are done
# USE_BASIC_HTTP = True # USE_BASIC_HTTP = True
@ -268,6 +232,35 @@ ADMIN_TOKEN = ""
# uncomment the two lines related to htpasswd # uncomment the two lines related to htpasswd
CKEDITOR_CONFIGS = {
"default": {
"toolbar": "Custom",
"width": "100%",
"toolbar_Custom": [
["Styles"],
["Format"],
["Bold", "Italic", "Underline"],
["HorizontalRule"],
[
"NumberedList",
"BulletedList",
"-",
"Outdent",
"Indent",
"-",
"JustifyLeft",
"JustifyCenter",
"JustifyRight",
"JustifyBlock",
],
["Link", "Unlink"],
["Image"],
["RemoveFormat", "Source"],
],
}
}
AUTH_USER_MODEL = "users.User" AUTH_USER_MODEL = "users.User"
LOGIN_REDIRECT_URL = "/" LOGIN_REDIRECT_URL = "/"
@ -277,7 +270,7 @@ AUTHENTICATION_BACKENDS = (
) )
INSTALLED_APPS = [ INSTALLED_APPS = [
"admin_customizations", "django.contrib.admin",
"django.contrib.auth", "django.contrib.auth",
"allauth", "allauth",
"allauth.account", "allauth.account",
@ -286,8 +279,6 @@ INSTALLED_APPS = [
"django.contrib.sessions", "django.contrib.sessions",
"django.contrib.messages", "django.contrib.messages",
"django.contrib.staticfiles", "django.contrib.staticfiles",
"jazzmin",
"django.contrib.admin",
"django.contrib.sites", "django.contrib.sites",
"rest_framework", "rest_framework",
"rest_framework.authtoken", "rest_framework.authtoken",
@ -295,31 +286,23 @@ INSTALLED_APPS = [
"files.apps.FilesConfig", "files.apps.FilesConfig",
"users.apps.UsersConfig", "users.apps.UsersConfig",
"actions.apps.ActionsConfig", "actions.apps.ActionsConfig",
"rbac.apps.RbacConfig",
"identity_providers.apps.IdentityProvidersConfig",
"debug_toolbar", "debug_toolbar",
"mptt", "mptt",
"crispy_forms", "crispy_forms",
"crispy_bootstrap5",
"uploader.apps.UploaderConfig", "uploader.apps.UploaderConfig",
"djcelery_email", "djcelery_email",
"drf_yasg", "ckeditor",
"allauth.socialaccount.providers.saml",
"saml_auth.apps.SamlAuthConfig",
"tinymce",
] ]
MIDDLEWARE = [ MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware", "django.middleware.security.SecurityMiddleware",
"django.contrib.sessions.middleware.SessionMiddleware", "django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.locale.LocaleMiddleware",
"django.middleware.common.CommonMiddleware", "django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware", "django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware", "django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware", "django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware", "django.middleware.clickjacking.XFrameOptionsMiddleware",
"debug_toolbar.middleware.DebugToolbarMiddleware", "debug_toolbar.middleware.DebugToolbarMiddleware",
"allauth.account.middleware.AccountMiddleware",
] ]
ROOT_URLCONF = "cms.urls" ROOT_URLCONF = "cms.urls"
@ -347,15 +330,11 @@ WSGI_APPLICATION = "cms.wsgi.application"
AUTH_PASSWORD_VALIDATORS = [ AUTH_PASSWORD_VALIDATORS = [
{ {
"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator", "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
"OPTIONS": {
"user_attributes": ("username", "email", "first_name", "last_name"),
"max_similarity": 0.7,
},
}, },
{ {
"NAME": "django.contrib.auth.password_validation.MinimumLengthValidator", "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
"OPTIONS": { "OPTIONS": {
"min_length": 7, "min_length": 5,
}, },
}, },
{ {
@ -369,16 +348,6 @@ FILE_UPLOAD_HANDLERS = [
LOGS_DIR = os.path.join(BASE_DIR, "logs") LOGS_DIR = os.path.join(BASE_DIR, "logs")
error_filename = os.path.join(LOGS_DIR, "debug.log")
if not os.path.exists(LOGS_DIR):
try:
os.mkdir(LOGS_DIR)
except PermissionError:
pass
if not os.path.isfile(error_filename):
open(error_filename, 'a').close()
LOGGING = { LOGGING = {
"version": 1, "version": 1,
"disable_existing_loggers": False, "disable_existing_loggers": False,
@ -386,7 +355,7 @@ LOGGING = {
"file": { "file": {
"level": "ERROR", "level": "ERROR",
"class": "logging.FileHandler", "class": "logging.FileHandler",
"filename": error_filename, "filename": os.path.join(LOGS_DIR, "debug.log"),
}, },
}, },
"loggers": { "loggers": {
@ -398,7 +367,16 @@ LOGGING = {
}, },
} }
DATABASES = {"default": {"ENGINE": "django.db.backends.postgresql", "NAME": "mediacms", "HOST": "127.0.0.1", "PORT": "5432", "USER": "mediacms", "PASSWORD": "mediacms", "OPTIONS": {'pool': True}}} DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "mediacms",
"HOST": "127.0.0.1",
"PORT": "5432",
"USER": "mediacms",
"PASSWORD": "mediacms",
}
}
REDIS_LOCATION = "redis://127.0.0.1:6379/1" REDIS_LOCATION = "redis://127.0.0.1:6379/1"
@ -445,155 +423,11 @@ CELERY_BEAT_SCHEDULE = {
# TODO: beat, delete chunks from media root # TODO: beat, delete chunks from media root
# chunks_dir after xx days...(also uploads_dir) # chunks_dir after xx days...(also uploads_dir)
LOCAL_INSTALL = False LOCAL_INSTALL = False
# this is an option to make the whole portal available to logged in users only
# it is placed here so it can be overrided on local_settings.py
GLOBAL_LOGIN_REQUIRED = False
# TODO: separate settings on production/development more properly, for now
# this should be ok
CELERY_TASK_ALWAYS_EAGER = False
if os.environ.get("TESTING"):
CELERY_TASK_ALWAYS_EAGER = True
# if True, only show original, don't perform any action on videos
DO_NOT_TRANSCODE_VIDEO = False
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
LANGUAGES = [
('ar', _('Arabic')),
('bn', _('Bengali')),
('da', _('Danish')),
('nl', _('Dutch')),
('en', _('English')),
('fr', _('French')),
('de', _('German')),
('hi', _('Hindi')),
('id', _('Indonesian')),
('it', _('Italian')),
('ja', _('Japanese')),
('ko', _('Korean')),
('pt', _('Portuguese')),
('ru', _('Russian')),
('zh-hans', _('Simplified Chinese')),
('sl', _('Slovenian')),
('zh-hant', _('Traditional Chinese')),
('es', _('Spanish')),
('tr', _('Turkish')),
('el', _('Greek')),
('ur', _('Urdu')),
('he', _('Hebrew')),
]
LANGUAGE_CODE = 'en' # default language
TINYMCE_DEFAULT_CONFIG = {
"theme": "silver",
"height": 500,
"resize": "both",
"menubar": "file edit view insert format tools table help",
"menu": {
"format": {
"title": "Format",
"items": "blocks | bold italic underline strikethrough superscript subscript code | " "fontfamily fontsize align lineheight | " "forecolor backcolor removeformat",
},
},
"plugins": "advlist,autolink,autosave,lists,link,image,charmap,print,preview,anchor,"
"searchreplace,visualblocks,code,fullscreen,insertdatetime,media,table,paste,directionality,"
"code,help,wordcount,emoticons,file,image,media",
"toolbar": "undo redo | code preview | blocks | "
"bold italic | alignleft aligncenter "
"alignright alignjustify ltr rtl | bullist numlist outdent indent | "
"removeformat | restoredraft help | image media",
"branding": False, # remove branding
"promotion": False, # remove promotion
"body_class": "page-main-inner custom-page-wrapper", # class of the body element in tinymce
"block_formats": "Paragraph=p; Heading 1=h1; Heading 2=h2; Heading 3=h3;",
"formats": { # customize h2 to always have emphasis-large class
"h2": {"block": "h2", "classes": "emphasis-large"},
},
"font_size_formats": "16px 18px 24px 32px",
"images_upload_url": "/tinymce/upload/",
"images_upload_handler": "tinymce.views.upload_image",
"automatic_uploads": True,
"file_picker_types": "image",
"paste_data_images": True,
"paste_as_text": False,
"paste_enable_default_filters": True,
"paste_word_valid_elements": "b,strong,i,em,h1,h2,h3,h4,h5,h6,p,br,a,ul,ol,li",
"paste_retain_style_properties": "all",
"paste_remove_styles": False,
"paste_merge_formats": True,
"sandbox_iframes": False,
}
SPRITE_NUM_SECS = 10
# number of seconds for sprite image.
# If you plan to change this, you must also follow the instructions on admins_docs.md
# to change the equivalent value in ./frontend/src/static/js/components/media-viewer/VideoViewer/index.js and then re-build frontend
# how many images will be shown on the slideshow
SLIDESHOW_ITEMS = 30
# this calculation is redundant most probably, setting as an option
CALCULATE_MD5SUM = False
CRISPY_ALLOWED_TEMPLATE_PACKS = "bootstrap5"
CRISPY_TEMPLATE_PACK = "bootstrap5"
# allow option to override the default admin url
# keep the trailing slash
DJANGO_ADMIN_URL = "admin/"
# this are used around a number of places and will need to be well documented!!!
USE_SAML = False
USE_RBAC = False
USE_IDENTITY_PROVIDERS = False
JAZZMIN_UI_TWEAKS = {"theme": "flatly"}
USE_ROUNDED_CORNERS = True
ALLOW_VIDEO_TRIMMER = True
ALLOW_CUSTOM_MEDIA_URLS = False
# Whether to allow anonymous users to list all users
ALLOW_ANONYMOUS_USER_LISTING = True
# Who can see the members page
# valid choices are all, editors, admins
CAN_SEE_MEMBERS_PAGE = "all"
# Maximum number of media a user can upload
NUMBER_OF_MEDIA_USER_CAN_UPLOAD = 100
# ffmpeg options
FFMPEG_DEFAULT_PRESET = "medium" # see https://trac.ffmpeg.org/wiki/Encode/H.264
# If 'all' is in the list, no check is performed
ALLOWED_MEDIA_UPLOAD_TYPES = ["video", "audio", "image", "pdf"]
# transcription options
# the mediacms-full docker image needs to be used in order to be able to use transcription
# if you are using the mediacms-full image, change USE_WHISPER_TRANSCRIBE to True
USE_WHISPER_TRANSCRIBE = False
# by default all users can request a video to be transcribed. If you want to
# allow only editors, set this to False
USER_CAN_TRANSCRIBE_VIDEO = True
# Whisper transcribe options - https://github.com/openai/whisper
WHISPER_MODEL = "base"
# show a custom text in the sidebar footer, otherwise the default will be shown if this is empty
SIDEBAR_FOOTER_TEXT = ""
try: try:
# keep a local_settings.py file for local overrides # keep a local_settings.py file for local overrides
from .local_settings import * # noqa from .local_settings import *
# ALLOWED_HOSTS needs a url/ip # ALLOWED_HOSTS needs a url/ip
ALLOWED_HOSTS.append(FRONTEND_HOST.replace("http://", "").replace("https://", "")) ALLOWED_HOSTS.append(FRONTEND_HOST.replace("http://", "").replace("https://", ""))
@ -601,43 +435,12 @@ except ImportError:
# local_settings not in use # local_settings not in use
pass pass
# Don't add new settings below that could be overridden in local_settings.py!!!
if "http" not in FRONTEND_HOST: if "http" not in FRONTEND_HOST:
# FRONTEND_HOST needs a http:// preffix # FRONTEND_HOST needs a http:// preffix
FRONTEND_HOST = f"http://{FRONTEND_HOST}" # noqa FRONTEND_HOST = f"http://{FRONTEND_HOST}"
if LOCAL_INSTALL: if LOCAL_INSTALL:
SSL_FRONTEND_HOST = FRONTEND_HOST.replace("http", "https") SSL_FRONTEND_HOST = FRONTEND_HOST.replace("http", "https")
else: else:
SSL_FRONTEND_HOST = FRONTEND_HOST SSL_FRONTEND_HOST = FRONTEND_HOST
# CSRF_COOKIE_SECURE = True
# SESSION_COOKIE_SECURE = True
PYSUBS_COMMAND = "pysubs2"
# the following is related to local development using docker
# and docker-compose-dev.yaml
try:
DEVELOPMENT_MODE = os.environ.get("DEVELOPMENT_MODE")
if DEVELOPMENT_MODE:
# keep a dev_settings.py file for local overrides
from .dev_settings import * # noqa
except ImportError:
pass
if GLOBAL_LOGIN_REQUIRED:
auth_index = MIDDLEWARE.index("django.contrib.auth.middleware.AuthenticationMiddleware")
MIDDLEWARE.insert(auth_index + 1, "django.contrib.auth.middleware.LoginRequiredMiddleware")
if USERS_NEEDS_TO_BE_APPROVED:
AUTHENTICATION_BACKENDS = (
'cms.auth_backends.ApprovalBackend',
'allauth.account.auth_backends.AuthenticationBackend',
)
auth_index = MIDDLEWARE.index("django.contrib.auth.middleware.AuthenticationMiddleware")
MIDDLEWARE.insert(auth_index + 1, "cms.middleware.ApprovalMiddleware")

View File

@ -1,38 +1,13 @@
import debug_toolbar
from django.conf import settings
from django.conf.urls import include
from django.contrib import admin from django.contrib import admin
from django.urls import path, re_path from django.urls import path
from django.views.generic.base import TemplateView from django.conf.urls import url, include
from drf_yasg import openapi import debug_toolbar
from drf_yasg.views import get_schema_view
from rest_framework.permissions import AllowAny
schema_view = get_schema_view(
openapi.Info(title="MediaCMS API", default_version='v1', contact=openapi.Contact(url="https://mediacms.io"), x_logo={"url": "../../static/images/logo_dark.svg"}),
public=True,
permission_classes=(AllowAny,),
)
# refactor seriously
urlpatterns = [ urlpatterns = [
re_path(r"^__debug__/", include(debug_toolbar.urls)), url(r"^__debug__/", include(debug_toolbar.urls)),
path( url(r"^", include("files.urls")),
"robots.txt", url(r"^", include("users.urls")),
TemplateView.as_view(template_name="robots.txt", content_type="text/plain"), url(r"^accounts/", include("allauth.urls")),
), url(r"^api-auth/", include("rest_framework.urls")),
re_path(r"^", include("files.urls")), path("admin/", admin.site.urls),
re_path(r"^", include("users.urls")),
re_path(r"^accounts/", include("allauth.urls")),
re_path(r"^api-auth/", include("rest_framework.urls")),
path(settings.DJANGO_ADMIN_URL, admin.site.urls),
re_path(r'^swagger(?P<format>\.json|\.yaml)$', schema_view.without_ui(cache_timeout=0), name='schema-json'),
re_path(r'^swagger/$', schema_view.with_ui('swagger', cache_timeout=0), name='schema-swagger-ui'),
path('docs/api/', schema_view.with_ui('redoc', cache_timeout=0), name='schema-redoc'),
path("tinymce/", include("tinymce.urls")),
] ]
admin.site.site_header = "MediaCMS Admin"
admin.site.site_title = "MediaCMS"
admin.site.index_title = "Admin"

View File

@ -1 +0,0 @@
VERSION = "7.0.1-beta.8"

View File

@ -1,75 +0,0 @@
# MediaCMS: Document Changes for DEIC
## Configuration Changes
The following changes are required in `deploy/docker/local_settings.py`:
```python
# default workflow
PORTAL_WORKFLOW = 'private'
# Authentication Settings
# these two are necessary so that users cannot register through system accounts. They can only register through identity providers
REGISTER_ALLOWED = False
USERS_CAN_SELF_REGISTER = False
USE_RBAC = True
USE_SAML = True
USE_IDENTITY_PROVIDERS = True
# Proxy and SSL Settings
USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SECURE_SSL_REDIRECT = True
CSRF_COOKIE_SECURE = True
SESSION_COOKIE_SECURE = True
# SAML Configuration
SOCIALACCOUNT_ADAPTER = 'saml_auth.adapter.SAMLAccountAdapter'
ACCOUNT_USERNAME_VALIDATORS = "users.validators.less_restrictive_username_validators"
SOCIALACCOUNT_PROVIDERS = {
"saml": {
"provider_class": "saml_auth.custom.provider.CustomSAMLProvider",
}
}
SOCIALACCOUNT_AUTO_SIGNUP = True
SOCIALACCOUNT_EMAIL_REQUIRED = False
# if set to strict, user is created with the email from the saml provider without
# checking if the email is already on the system
# however if this is ommited, and user tries to login with an email that already exists on
# the system, then they get to the ugly form where it suggests they add a username/email/name
ACCOUNT_PREVENT_ENUMERATION = 'strict'
```
## SAML Configuration Steps
### Step 1: Add SAML Identity Provider
1. Navigate to Admin panel
2. Select "Identity Provider"
3. Configure as follows:
- **Provider**: saml # ensure this is set with lower case!
- **Provider ID**: `wayf.wayf.dk`
- **IDP Config Name**: `Deic` (or preferred name)
- **Client ID**: `wayf_dk` (important: defines the URL, e.g., `https://deic.mediacms.io/accounts/saml/wayf_dk`)
- **Site**: Set the default one
### Step 2: Add SAML Configuration
Can be set through the SAML Configurations tab:
1. **IDP ID**: Must be a URL, e.g., `https://wayf.wayf.dk`
2. **IDP Certificate**: x509cert from your SAML provider
3. **SSO URL**: `https://wayf.wayf.dk/saml2/idp/SSOService2.php`
4. **SLO URL**: `https://wayf.wayf.dk/saml2/idp/SingleLogoutService.php`
5. **SP Metadata URL**: The metadata URL set for the SP, e.g., `https://deic.mediacms.io/saml/metadata`. This should point to the URL of the SP and is autogenerated
### Step 3: Set the other Options
1. **Email Settings**:
- `verified_email`: When enabled, emails from SAML responses will be marked as verified
- `Remove from groups`: When enabled, user is removed from a group after login, if they have been removed from the group on the IDP
2. **Global Role Mapping**: Maps the role returned by SAML (as set in the SAML Configuration tab) with the role in MediaCMS
3. **Group Role Mapping**: Maps the role returned by SAML (as set in the SAML Configuration tab) with the role in groups that user will be added
4. **Group mapping**: This creates groups associated with this IDP. Group ids as they come from SAML, associated with MediaCMS groups
5. **Category Mapping**: This maps a group id (from SAML response) with a category in MediaCMS

View File

@ -7,12 +7,11 @@ ln -sf /dev/stdout /var/log/nginx/mediacms.io.access.log && ln -sf /dev/stderr /
cp /home/mediacms.io/mediacms/deploy/docker/local_settings.py /home/mediacms.io/mediacms/cms/local_settings.py cp /home/mediacms.io/mediacms/deploy/docker/local_settings.py /home/mediacms.io/mediacms/cms/local_settings.py
mkdir -p /home/mediacms.io/mediacms/{logs,pids,media_files/hls}
mkdir -p /home/mediacms.io/mediacms/{logs,media_files/hls}
touch /home/mediacms.io/mediacms/logs/debug.log touch /home/mediacms.io/mediacms/logs/debug.log
mkdir -p /var/run/mediacms # Remove any dangling pids
chown www-data:www-data /var/run/mediacms rm -rf /home/mediacms.io/mediacms/pids/*
TARGET_GID=$(stat -c "%g" /home/mediacms.io/mediacms/) TARGET_GID=$(stat -c "%g" /home/mediacms.io/mediacms/)
@ -29,9 +28,7 @@ else
fi fi
# We should do this only for folders that have a different owner, since it is an expensive operation # We should do this only for folders that have a different owner, since it is an expensive operation
# Also ignoring .git folder to fix this issue https://github.com/mediacms-io/mediacms/issues/934 find /home/mediacms.io/ ! \( -user www-data -group $TARGET_GID \) -exec chown www-data:$TARGET_GID {} +
# Exclude package-lock.json files that may not exist or be removed during frontend setup
find /home/mediacms.io/mediacms ! \( -path "*.git*" -o -name "package-lock.json" \) -exec chown www-data:$TARGET_GID {} + 2>/dev/null || true
chmod +x /home/mediacms.io/mediacms/deploy/docker/start.sh /home/mediacms.io/mediacms/deploy/docker/prestart.sh chmod +x /home/mediacms.io/mediacms/deploy/docker/start.sh /home/mediacms.io/mediacms/deploy/docker/prestart.sh

View File

@ -1,19 +1,17 @@
import os FRONTEND_HOST = 'http://localhost'
PORTAL_NAME = 'MediaCMS'
FRONTEND_HOST = os.getenv('FRONTEND_HOST', 'http://localhost') SECRET_KEY = 'ma!s3^b-cw!f#7s6s0m3*jx77a@riw(7701**(r=ww%w!2+yk2'
PORTAL_NAME = os.getenv('PORTAL_NAME', 'MediaCMS') POSTGRES_HOST = 'db'
SECRET_KEY = os.getenv('SECRET_KEY', 'ma!s3^b-cw!f#7s6s0m3*jx77a@riw(7701**(r=ww%w!2+yk2') REDIS_LOCATION = "redis://redis:6379/1"
REDIS_LOCATION = os.getenv('REDIS_LOCATION', 'redis://redis:6379/1')
DATABASES = { DATABASES = {
"default": { "default": {
"ENGINE": "django.db.backends.postgresql", "ENGINE": "django.db.backends.postgresql",
"NAME": os.getenv('POSTGRES_NAME', 'mediacms'), "NAME": "mediacms",
"HOST": os.getenv('POSTGRES_HOST', 'db'), "HOST": POSTGRES_HOST,
"PORT": os.getenv('POSTGRES_PORT', '5432'), "PORT": "5432",
"USER": os.getenv('POSTGRES_USER', 'mediacms'), "USER": "mediacms",
"PASSWORD": os.getenv('POSTGRES_PASSWORD', 'mediacms'), "PASSWORD": "mediacms",
"OPTIONS": {'pool': True},
} }
} }
@ -31,6 +29,8 @@ CACHES = {
BROKER_URL = REDIS_LOCATION BROKER_URL = REDIS_LOCATION
CELERY_RESULT_BACKEND = BROKER_URL CELERY_RESULT_BACKEND = BROKER_URL
MP4HLS_COMMAND = "/home/mediacms.io/bento4/bin/mp4hls" MP4HLS_COMMAND = (
"/home/mediacms.io/bento4/bin/mp4hls"
)
DEBUG = os.getenv('DEBUG', 'False') == 'True' DEBUG = False

View File

@ -16,10 +16,6 @@ server {
location /media { location /media {
alias /home/mediacms.io/mediacms/media_files ; alias /home/mediacms.io/mediacms/media_files ;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
} }
location / { location / {

View File

@ -1,99 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE policymap [
<!ELEMENT policymap (policy)*>
<!ATTLIST policymap xmlns CDATA #FIXED ''>
<!ELEMENT policy EMPTY>
<!ATTLIST policy xmlns CDATA #FIXED '' domain NMTOKEN #REQUIRED
name NMTOKEN #IMPLIED pattern CDATA #IMPLIED rights NMTOKEN #IMPLIED
stealth NMTOKEN #IMPLIED value CDATA #IMPLIED>
]>
<!--
Configure ImageMagick policies.
Domains include system, delegate, coder, filter, path, or resource.
Rights include none, read, write, execute and all. Use | to combine them,
for example: "read | write" to permit read from, or write to, a path.
Use a glob expression as a pattern.
Suppose we do not want users to process MPEG video images:
<policy domain="delegate" rights="none" pattern="mpeg:decode" />
Here we do not want users reading images from HTTP:
<policy domain="coder" rights="none" pattern="HTTP" />
The /repository file system is restricted to read only. We use a glob
expression to match all paths that start with /repository:
<policy domain="path" rights="read" pattern="/repository/*" />
Lets prevent users from executing any image filters:
<policy domain="filter" rights="none" pattern="*" />
Any large image is cached to disk rather than memory:
<policy domain="resource" name="area" value="1GP"/>
Use the default system font unless overwridden by the application:
<policy domain="system" name="font" value="/usr/share/fonts/favorite.ttf"/>
Define arguments for the memory, map, area, width, height and disk resources
with SI prefixes (.e.g 100MB). In addition, resource policies are maximums
for each instance of ImageMagick (e.g. policy memory limit 1GB, -limit 2GB
exceeds policy maximum so memory limit is 1GB).
Rules are processed in order. Here we want to restrict ImageMagick to only
read or write a small subset of proven web-safe image types:
<policy domain="delegate" rights="none" pattern="*" />
<policy domain="filter" rights="none" pattern="*" />
<policy domain="coder" rights="none" pattern="*" />
<policy domain="coder" rights="read|write" pattern="{GIF,JPEG,PNG,WEBP}" />
-->
<policymap>
<!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
<policy domain="resource" name="memory" value="1GiB"/>
<policy domain="resource" name="map" value="30GiB"/>
<policy domain="resource" name="width" value="16MP"/>
<policy domain="resource" name="height" value="16MP"/>
<!-- <policy domain="resource" name="list-length" value="128"/> -->
<policy domain="resource" name="area" value="40GP"/>
<policy domain="resource" name="disk" value="100GiB"/>
<!-- <policy domain="resource" name="file" value="768"/> -->
<!-- <policy domain="resource" name="thread" value="4"/> -->
<!-- <policy domain="resource" name="throttle" value="0"/> -->
<!-- <policy domain="resource" name="time" value="3600"/> -->
<!-- <policy domain="coder" rights="none" pattern="MVG" /> -->
<!-- <policy domain="module" rights="none" pattern="{PS,PDF,XPS}" /> -->
<!-- <policy domain="path" rights="none" pattern="@*" /> -->
<!-- <policy domain="cache" name="memory-map" value="anonymous"/> -->
<!-- <policy domain="cache" name="synchronize" value="True"/> -->
<!-- <policy domain="cache" name="shared-secret" value="passphrase" stealth="true"/>
<!-- <policy domain="system" name="max-memory-request" value="256MiB"/> -->
<!-- <policy domain="system" name="shred" value="2"/> -->
<!-- <policy domain="system" name="precision" value="6"/> -->
<!-- <policy domain="system" name="font" value="/path/to/font.ttf"/> -->
<!-- <policy domain="system" name="pixel-cache-memory" value="anonymous"/> -->
<!-- <policy domain="system" name="shred" value="2"/> -->
<!-- <policy domain="system" name="precision" value="6"/> -->
<!-- not needed due to the need to use explicitly by mvg: -->
<!-- <policy domain="delegate" rights="none" pattern="MVG" /> -->
<!-- use curl -->
<policy domain="delegate" rights="none" pattern="URL" />
<policy domain="delegate" rights="none" pattern="HTTPS" />
<policy domain="delegate" rights="none" pattern="HTTP" />
<!-- in order to avoid to get image with password text -->
<policy domain="path" rights="none" pattern="@*"/>
<!-- disable ghostscript format types -->
<policy domain="coder" rights="none" pattern="PS" />
<policy domain="coder" rights="none" pattern="PS2" />
<policy domain="coder" rights="none" pattern="PS3" />
<policy domain="coder" rights="none" pattern="EPS" />
<policy domain="coder" rights="none" pattern="PDF" />
<policy domain="coder" rights="none" pattern="XPS" />
</policymap>

View File

@ -4,29 +4,20 @@ RANDOM_ADMIN_PASS=`python -c "import secrets;chars = 'abcdefghijklmnopqrstuvwxyz
ADMIN_PASSWORD=${ADMIN_PASSWORD:-$RANDOM_ADMIN_PASS} ADMIN_PASSWORD=${ADMIN_PASSWORD:-$RANDOM_ADMIN_PASS}
if [ X"$ENABLE_MIGRATIONS" = X"yes" ]; then if [ X"$ENABLE_MIGRATIONS" = X"yes" ]; then
echo "Running migrations service"
python manage.py migrate python manage.py migrate
EXISTING_INSTALLATION=`echo "from users.models import User; print(User.objects.exists())" |python manage.py shell` python manage.py loaddata fixtures/encoding_profiles.json
if [ "$EXISTING_INSTALLATION" = "True" ]; then python manage.py loaddata fixtures/categories.json
echo "Loaddata has already run"
else
echo "Running loaddata and creating admin user"
python manage.py loaddata fixtures/encoding_profiles.json
python manage.py loaddata fixtures/categories.json
# post_save, needs redis to succeed (ie. migrate depends on redis)
DJANGO_SUPERUSER_PASSWORD=$ADMIN_PASSWORD python manage.py createsuperuser \
--no-input \
--username=$ADMIN_USER \
--email=$ADMIN_EMAIL \
--database=default || true
echo "Created admin user with password: $ADMIN_PASSWORD"
fi
echo "RUNNING COLLECTSTATIC"
python manage.py collectstatic --noinput python manage.py collectstatic --noinput
echo "Admin Password: $ADMIN_PASSWORD"
# post_save, needs redis to succeed (ie. migrate depends on redis)
DJANGO_SUPERUSER_PASSWORD=$ADMIN_PASSWORD python manage.py createsuperuser \
--no-input \
--username=$ADMIN_USER \
--email=$ADMIN_EMAIL \
--database=default || true
# echo "Updating hostname ..." # echo "Updating hostname ..."
# TODO: Get the FRONTEND_HOST from cms/local_settings.py # TODO: Get the FRONTEND_HOST from cms/local_settings.py
# echo "from django.contrib.sites.models import Site; Site.objects.update(name='$FRONTEND_HOST', domain='$FRONTEND_HOST')" | python manage.py shell # echo "from django.contrib.sites.models import Site; Site.objects.update(name='$FRONTEND_HOST', domain='$FRONTEND_HOST')" | python manage.py shell
@ -67,5 +58,4 @@ fi
if [ X"$ENABLE_CELERY_LONG" = X"yes" ] ; then if [ X"$ENABLE_CELERY_LONG" = X"yes" ] ; then
echo "Enabling celery-long task worker" echo "Enabling celery-long task worker"
cp deploy/docker/supervisord/supervisord-celery_long.conf /etc/supervisor/conf.d/supervisord-celery_long.conf cp deploy/docker/supervisord/supervisord-celery_long.conf /etc/supervisor/conf.d/supervisord-celery_long.conf
rm /var/run/mediacms/* -f # remove any stale id, so that on forced restarts of celery workers there are no stale processes that prevent new ones
fi fi

View File

@ -1 +1 @@
client_max_body_size 5800M; client_max_body_size 1g;

View File

@ -9,4 +9,4 @@ user=www-data
directory=/home/mediacms.io/mediacms directory=/home/mediacms.io/mediacms
priority=300 priority=300
startinorder=true startinorder=true
command=/home/mediacms.io/bin/celery beat --pidfile=/var/run/mediacms/beat%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_beat.log command=/home/mediacms.io/bin/celery beat --pidfile=/home/mediacms.io/mediacms/pids/beat%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_beat.log

View File

@ -10,4 +10,4 @@ directory=/home/mediacms.io/mediacms
priority=500 priority=500
startinorder=true startinorder=true
startsecs=0 startsecs=0
command=/home/mediacms.io/bin/celery multi start long1 --pidfile=/var/run/mediacms/%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_long.log -Ofair --prefetch-multiplier=1 -Q long_tasks command=/home/mediacms.io/bin/celery multi start long1 --pidfile=/home/mediacms.io/mediacms/pids/%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_long.log -Ofair --prefetch-multiplier=1 -Q long_tasks

View File

@ -9,4 +9,4 @@ user=www-data
directory=/home/mediacms.io/mediacms directory=/home/mediacms.io/mediacms
priority=400 priority=400
startinorder=true startinorder=true
command=/home/mediacms.io/bin/celery multi start short1 short2 --pidfile=/var/run/mediacms/%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_short.log --soft-time-limit=300 -c10 -Q short_tasks command=/home/mediacms.io/bin/celery multi start short1 short2 --pidfile=/home/mediacms.io/mediacms/pids/%%n.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_short.log --soft-time-limit=300 -c10 -Q short_tasks

View File

@ -21,4 +21,3 @@ vacuum = true
hook-master-start = unix_signal:15 gracefully_kill_them_all hook-master-start = unix_signal:15 gracefully_kill_them_all
need-app = true need-app = true
die-on-term = true die-on-term = true
buffer-size=32768

View File

@ -8,13 +8,15 @@ User=www-data
Group=www-data Group=www-data
Restart=always Restart=always
RestartSec=10 RestartSec=10
WorkingDirectory=/home/mediacms.io/mediacms Environment=APP_DIR="/home/mediacms.io/mediacms"
Environment=CELERY_BIN="/home/mediacms.io/bin/celery" Environment=CELERY_BIN="/home/mediacms.io/bin/celery"
Environment=CELERY_APP="cms"
Environment=CELERYD_PID_FILE="/home/mediacms.io/mediacms/pids/beat%n.pid" Environment=CELERYD_PID_FILE="/home/mediacms.io/mediacms/pids/beat%n.pid"
Environment=CELERYD_LOG_FILE="/home/mediacms.io/mediacms/logs/beat%N.log" Environment=CELERYD_LOG_FILE="/home/mediacms.io/mediacms/logs/beat%N.log"
Environment=CELERYD_LOG_LEVEL="INFO" Environment=CELERYD_LOG_LEVEL="INFO"
Environment=APP_DIR="/home/mediacms.io/mediacms"
ExecStart=/bin/sh -c '${CELERY_BIN} -A cms beat --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}' ExecStart=/bin/sh -c '${CELERY_BIN} beat -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} --workdir=${APP_DIR}'
ExecStop=/bin/kill -s TERM $MAINPID ExecStop=/bin/kill -s TERM $MAINPID
[Install] [Install]

View File

@ -8,21 +8,23 @@ User=www-data
Group=www-data Group=www-data
Restart=always Restart=always
RestartSec=10 RestartSec=10
WorkingDirectory=/home/mediacms.io/mediacms Environment=APP_DIR="/home/mediacms.io/mediacms"
Environment=CELERYD_NODES="long1" Environment=CELERYD_NODES="long1"
Environment=CELERY_QUEUE="long_tasks" Environment=CELERY_QUEUE="long_tasks"
Environment=CELERY_BIN="/home/mediacms.io/bin/celery" Environment=CELERY_BIN="/home/mediacms.io/bin/celery"
Environment=CELERY_APP="cms"
Environment=CELERYD_MULTI="multi" Environment=CELERYD_MULTI="multi"
Environment=CELERYD_OPTS="-Ofair --prefetch-multiplier=1" Environment=CELERYD_OPTS="-Ofair --prefetch-multiplier=1"
Environment=CELERYD_PID_FILE="/home/mediacms.io/mediacms/pids/%n.pid" Environment=CELERYD_PID_FILE="/home/mediacms.io/mediacms/pids/%n.pid"
Environment=CELERYD_LOG_FILE="/home/mediacms.io/mediacms/logs/%N.log" Environment=CELERYD_LOG_FILE="/home/mediacms.io/mediacms/logs/%N.log"
Environment=CELERYD_LOG_LEVEL="INFO" Environment=CELERYD_LOG_LEVEL="INFO"
Environment=APP_DIR="/home/mediacms.io/mediacms"
ExecStart=/bin/sh -c '${CELERY_BIN} -A cms multi start ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} -Q ${CELERY_QUEUE}' ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} --workdir=${APP_DIR} -Q ${CELERY_QUEUE}'
ExecStop=/bin/sh -c '${CELERY_BIN} -A cms multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE}' ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} -A cms multi restart ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} -Q ${CELERY_QUEUE}' ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} --workdir=${APP_DIR} -Q ${CELERY_QUEUE}'
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target

View File

@ -8,13 +8,14 @@ User=www-data
Group=www-data Group=www-data
Restart=always Restart=always
RestartSec=10 RestartSec=10
WorkingDirectory=/home/mediacms.io/mediacms Environment=APP_DIR="/home/mediacms.io/mediacms"
Environment=CELERYD_NODES="short1 short2" Environment=CELERYD_NODES="short1 short2"
Environment=CELERY_QUEUE="short_tasks" Environment=CELERY_QUEUE="short_tasks"
# Absolute or relative path to the 'celery' command: # Absolute or relative path to the 'celery' command:
Environment=CELERY_BIN="/home/mediacms.io/bin/celery" Environment=CELERY_BIN="/home/mediacms.io/bin/celery"
# App instance to use # App instance to use
# comment out this line if you don't use an app # comment out this line if you don't use an app
Environment=CELERY_APP="cms"
# or fully qualified: # or fully qualified:
#CELERY_APP="proj.tasks:app" #CELERY_APP="proj.tasks:app"
# How to call manage.py # How to call manage.py
@ -27,12 +28,13 @@ Environment=CELERYD_OPTS="--soft-time-limit=300 -c10"
Environment=CELERYD_PID_FILE="/home/mediacms.io/mediacms/pids/%n.pid" Environment=CELERYD_PID_FILE="/home/mediacms.io/mediacms/pids/%n.pid"
Environment=CELERYD_LOG_FILE="/home/mediacms.io/mediacms/logs/%N.log" Environment=CELERYD_LOG_FILE="/home/mediacms.io/mediacms/logs/%N.log"
Environment=CELERYD_LOG_LEVEL="INFO" Environment=CELERYD_LOG_LEVEL="INFO"
Environment=APP_DIR="/home/mediacms.io/mediacms"
ExecStart=/bin/sh -c '${CELERY_BIN} -A cms multi start ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} -Q ${CELERY_QUEUE}' ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} --workdir=${APP_DIR} -Q ${CELERY_QUEUE}'
ExecStop=/bin/sh -c '${CELERY_BIN} -A cms multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE}' ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} -A cms multi restart ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} -Q ${CELERY_QUEUE}' ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS} --workdir=${APP_DIR} -Q ${CELERY_QUEUE}'
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target

View File

@ -1,13 +0,0 @@
-----BEGIN DH PARAMETERS-----
MIICCAKCAgEAo3MMiEY/fNbu+usIM0cDi6x8G3JBApv0Lswta4kiyedWT1WN51iQ
9zhOFpmcu6517f/fR9MUdyhVKHxxSqWQTcmTEFtz4P3VLTS/W1N5VbKE2VEMLpIi
wr350aGvV1Er0ujcp5n4O4h0I1tn4/fNyDe7+pHCdwM+hxe8hJ3T0/tKtad4fnIs
WHDjl4f7m7KuFfheiK7Efb8MsT64HDDAYXn+INjtDZrbE5XPw20BqyWkrf07FcPx
8o9GW50Ox7/FYq7jVMI/skEu0BRc8u6uUD9+UOuWUQpdeHeFcvLOgW53Z03XwWuX
RXosUKzBPuGtUDAaKD/HsGW6xmGr2W9yRmu27jKpfYLUb/eWbbnRJwCw04LdzPqv
jmtq02Gioo3lf5H5wYV9IYF6M8+q/slpbttsAcKERimD1273FBRt5VhSugkXWKjr
XDhoXu6vZgj8Opei38qPa8pI1RUFoXHFlCe6WpZQmU8efL8gAMrJr9jUIY8eea1n
u20t5B9ueb9JMjrNafcq6QkKhZLi6fRDDTUyeDvc0dN9R/3Yts97SXfdi1/lX7HS
Ht4zXd5hEkvjo8GcnjsfZpAC39QfHWkDaeUGEqsl3jXjVMfkvoVY51OuokPWZzrJ
M5+wyXNpfGbH67dPk7iHgN7VJvgX0SYscDPTtms50Vk7RwEzLeGuSHMCAQI=
-----END DH PARAMETERS-----

View File

@ -46,12 +46,6 @@ server {
ssl_certificate_key /etc/letsencrypt/live/localhost/privkey.pem; ssl_certificate_key /etc/letsencrypt/live/localhost/privkey.pem;
ssl_certificate /etc/letsencrypt/live/localhost/fullchain.pem; ssl_certificate /etc/letsencrypt/live/localhost/fullchain.pem;
ssl_dhparam /etc/nginx/dhparams/dhparams.pem;
ssl_protocols TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_ecdh_curve secp521r1:secp384r1;
ssl_prefer_server_ciphers on;
gzip on; gzip on;
access_log /var/log/nginx/mediacms.io.access.log; access_log /var/log/nginx/mediacms.io.access.log;

View File

@ -19,7 +19,10 @@ http {
include /etc/nginx/mime.types; include /etc/nginx/mime.types;
default_type application/octet-stream; default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.log; access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log; error_log /var/log/nginx/error.log;

View File

@ -1,34 +0,0 @@
module selinux-mediacms 1.0;
require {
type init_t;
type var_t;
type redis_port_t;
type postgresql_port_t;
type httpd_t;
type httpd_sys_content_t;
type httpd_sys_rw_content_t;
class file { append create execute execute_no_trans getattr ioctl lock open read rename setattr unlink write };
class dir { add_name remove_name rmdir };
class tcp_socket name_connect;
class lnk_file read;
}
#============= httpd_t ==============
allow httpd_t var_t:file { getattr open read };
#============= init_t ==============
allow init_t postgresql_port_t:tcp_socket name_connect;
allow init_t redis_port_t:tcp_socket name_connect;
allow init_t httpd_sys_content_t:dir rmdir;
allow init_t httpd_sys_content_t:file { append create execute execute_no_trans ioctl lock open read rename setattr unlink write };
allow init_t httpd_sys_content_t:lnk_file read;
allow init_t httpd_sys_rw_content_t:dir { add_name remove_name rmdir };
allow init_t httpd_sys_rw_content_t:file { create ioctl lock open read setattr unlink write };

View File

@ -24,4 +24,4 @@ vacuum = true
logto = /home/mediacms.io/mediacms/logs/errorlog.txt logto = /home/mediacms.io/mediacms/logs/errorlog.txt
disable-logging = true disable-logging = true
buffer-size=32768

View File

@ -1,40 +0,0 @@
#!/bin/bash
# This script builds the video editor package and deploys the frontend assets to the static directory.
# How to run: sh deploy/scripts/build_and_deploy.sh
# Exit on any error
set -e
echo "Starting build process..."
# Build video editor package
echo "Building video editor package..."
cd frontend-tools/video-editor
yarn build:django
cd ../../
# Build chapter editor package
echo "Building chapters editor package..."
cd frontend-tools/chapters-editor
yarn build:django
cd ../../
# Build video js package
echo "Building video js package..."
cd frontend-tools/video-js
yarn build:django
cd ../../
# Run npm build in the frontend container
echo "Building frontend assets..."
docker compose -f docker-compose/docker-compose-dev-updated.yaml exec frontend npm run dist
# Copy static assets to the static directory
echo "Copying static assets..."
cp -r frontend/dist/static/* static/
# Restart the web service
echo "Restarting web service..."
docker compose -f docker-compose/docker-compose-dev-updated.yaml restart web
echo "Build and deployment completed successfully!"

View File

@ -1,89 +0,0 @@
version: "3"
services:
migrations:
build:
context: .
dockerfile: ./Dockerfile
target: base
args:
- DEVELOPMENT_MODE=True
image: mediacms/mediacms-dev:latest
volumes:
- ./:/home/mediacms.io/mediacms/
command: "./deploy/docker/prestart.sh"
environment:
DEVELOPMENT_MODE: True
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
ADMIN_PASSWORD: 'admin'
restart: on-failure
depends_on:
redis:
condition: service_healthy
db:
condition: service_healthy
frontend:
image: node:20
volumes:
- ${PWD}/frontend:/home/mediacms.io/mediacms/frontend/
working_dir: /home/mediacms.io/mediacms/frontend/
command: bash -c "npm install && npm run start"
env_file:
- ${PWD}/frontend/.env
ports:
- "8088:8088"
depends_on:
- web
web:
image: mediacms/mediacms-dev:latest
command: "python manage.py runserver 0.0.0.0:80"
environment:
DEVELOPMENT_MODE: True
ports:
- "80:80"
volumes:
- ./:/home/mediacms.io/mediacms/
depends_on:
- migrations
db:
image: postgres:17.2-alpine
volumes:
- ../postgres_data:/var/lib/postgresql/data/
restart: always
environment:
POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}", "--host=db", "--dbname=$POSTGRES_DB", "--username=$POSTGRES_USER"]
interval: 10s
timeout: 5s
retries: 5
redis:
image: "redis:alpine"
restart: always
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 30s
timeout: 10s
retries: 3
celery_worker:
image: mediacms/mediacms-dev:latest
deploy:
replicas: 1
volumes:
- ./:/home/mediacms.io/mediacms/
environment:
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_BEAT: 'no'
ENABLE_MIGRATIONS: 'no'
depends_on:
- web

View File

@ -18,11 +18,6 @@ services:
ENABLE_CELERY_SHORT: 'no' ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no' ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no' ENABLE_CELERY_BEAT: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
#ADMIN_PASSWORD: 'uncomment_and_set_password_here'
command: "./deploy/docker/prestart.sh"
restart: on-failure
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
@ -68,7 +63,7 @@ services:
depends_on: depends_on:
- migrations - migrations
db: db:
image: postgres:17.2-alpine image: postgres
volumes: volumes:
- ../postgres_data/:/var/lib/postgresql/data/ - ../postgres_data/:/var/lib/postgresql/data/
restart: always restart: always
@ -76,9 +71,8 @@ services:
POSTGRES_USER: mediacms POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck: healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}", "--host=db", "--dbname=$POSTGRES_DB", "--username=$POSTGRES_USER"] test: ["CMD-SHELL", "pg_isready -U mediacms"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 5

View File

@ -20,8 +20,6 @@ services:
ENABLE_CELERY_SHORT: 'no' ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no' ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no' ENABLE_CELERY_BEAT: 'no'
command: "./deploy/docker/prestart.sh"
restart: on-failure
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
@ -38,9 +36,6 @@ services:
ENABLE_CELERY_SHORT: 'no' ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no' ENABLE_CELERY_LONG: 'no'
ENABLE_MIGRATIONS: 'no' ENABLE_MIGRATIONS: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
#ADMIN_PASSWORD: 'uncomment_and_set_password_here'
VIRTUAL_HOST: localhost VIRTUAL_HOST: localhost
depends_on: depends_on:
- migrations - migrations
@ -70,7 +65,7 @@ services:
depends_on: depends_on:
- migrations - migrations
db: db:
image: postgres:17.2-alpine image: postgres
volumes: volumes:
- ../postgres_data/:/var/lib/postgresql/data/ - ../postgres_data/:/var/lib/postgresql/data/
restart: always restart: always
@ -78,9 +73,8 @@ services:
POSTGRES_USER: mediacms POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck: healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}", "--host=db", "--dbname=$POSTGRES_DB", "--username=$POSTGRES_USER"] test: ["CMD-SHELL", "pg_isready -U mediacms"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 5

View File

@ -11,11 +11,6 @@ services:
ENABLE_CELERY_SHORT: 'no' ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no' ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no' ENABLE_CELERY_BEAT: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
#ADMIN_PASSWORD: 'uncomment_and_set_password_here'
command: "./deploy/docker/prestart.sh"
restart: on-failure
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
@ -66,7 +61,7 @@ services:
depends_on: depends_on:
- migrations - migrations
db: db:
image: postgres:17.2-alpine image: postgres
volumes: volumes:
- postgres_data:/var/lib/postgresql/data/ - postgres_data:/var/lib/postgresql/data/
restart: always restart: always
@ -74,9 +69,8 @@ services:
POSTGRES_USER: mediacms POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck: healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}", "--host=db", "--dbname=$POSTGRES_DB", "--username=$POSTGRES_USER"] test: ["CMD-SHELL", "pg_isready -U mediacms"]
interval: 30s interval: 30s
timeout: 10s timeout: 10s
retries: 5 retries: 5

View File

@ -1,5 +0,0 @@
version: "3"
services:
celery_worker:
image: mediacms/mediacms:full

View File

@ -11,11 +11,6 @@ services:
ENABLE_CELERY_SHORT: 'no' ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no' ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no' ENABLE_CELERY_BEAT: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
# ADMIN_PASSWORD: 'uncomment_and_set_password_here'
command: "./deploy/docker/prestart.sh"
restart: on-failure
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
@ -62,7 +57,7 @@ services:
depends_on: depends_on:
- migrations - migrations
db: db:
image: postgres:17.2-alpine image: postgres
volumes: volumes:
- ../postgres_data:/var/lib/postgresql/data/ - ../postgres_data:/var/lib/postgresql/data/
restart: always restart: always
@ -70,9 +65,8 @@ services:
POSTGRES_USER: mediacms POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck: healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"] test: ["CMD-SHELL", "pg_isready -U mediacms"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 5
@ -81,6 +75,6 @@ services:
restart: always restart: always
healthcheck: healthcheck:
test: ["CMD", "redis-cli","ping"] test: ["CMD", "redis-cli","ping"]
interval: 10s interval: 30s
timeout: 5s timeout: 10s
retries: 3 retries: 3

View File

@ -1,124 +0,0 @@
name: mediacms-dev
services:
migrations:
platform: linux/amd64
build:
context: ..
dockerfile: Dockerfile
args:
- DEVELOPMENT_MODE=True
image: mediacms/mediacms:latest
volumes:
- ../:/home/mediacms.io/mediacms/
command: "/home/mediacms.io/mediacms/deploy/docker/prestart.sh"
environment:
DEVELOPMENT_MODE: True
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
ADMIN_PASSWORD: 'admin'
restart: on-failure
depends_on:
redis:
condition: service_healthy
db:
condition: service_healthy
frontend:
image: node:20
user: "root"
volumes:
- ${PWD}/frontend:/home/mediacms.io/mediacms/frontend/
- frontend_node_modules:/home/mediacms.io/mediacms/frontend/node_modules
- scripts_node_modules:/home/mediacms.io/mediacms/frontend/packages/scripts/node_modules
- npm_cache:/home/node/.npm
working_dir: /home/mediacms.io/mediacms/frontend/
command: >
bash -c "
echo 'Checking dependencies...' &&
if [ ! -f node_modules/.install-complete ]; then
echo 'First-time setup or dependencies changed, installing...' &&
npm install --legacy-peer-deps --cache /home/node/.npm &&
cd packages/scripts &&
npm install --legacy-peer-deps --cache /home/node/.npm &&
npm run build &&
cd ../.. &&
touch node_modules/.install-complete &&
echo 'Dependencies installed successfully'
else
echo 'Dependencies already installed, skipping installation...' &&
if [ ! -d packages/scripts/dist ]; then
echo 'Building scripts package...' &&
cd packages/scripts &&
npm run build &&
cd ../..
fi
fi &&
echo 'Starting development server...' &&
npm run start
"
env_file:
- ${PWD}/frontend/.env
ports:
- "8088:8088"
depends_on:
- web
restart: unless-stopped
web:
platform: linux/amd64
image: mediacms/mediacms:latest
command: "python manage.py runserver 0.0.0.0:80"
environment:
DEVELOPMENT_MODE: True
ports:
- "80:80"
volumes:
- ../:/home/mediacms.io/mediacms/
depends_on:
- migrations
db:
image: postgres:17.2-alpine
volumes:
- ../postgres_data:/var/lib/postgresql/data/
restart: always
environment:
POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}", "--host=db", "--dbname=$POSTGRES_DB", "--username=$POSTGRES_USER"]
interval: 10s
timeout: 5s
retries: 5
redis:
image: "redis:alpine"
restart: always
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 30s
timeout: 10s
retries: 3
celery_worker:
platform: linux/amd64
image: mediacms/mediacms:latest
deploy:
replicas: 1
volumes:
- ../:/home/mediacms.io/mediacms/
environment:
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_BEAT: 'no'
ENABLE_MIGRATIONS: 'no'
DEVELOPMENT_MODE: True
depends_on:
- web
volumes:
frontend_node_modules:
scripts_node_modules:
npm_cache:

View File

@ -1,121 +0,0 @@
version: "3"
# Uses https://github.com/nginx-proxy/acme-companion
services:
nginx-proxy:
image: nginxproxy/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- conf:/etc/nginx/conf.d
- vhost:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- dhparam:/etc/nginx/dhparam
- certs:/etc/nginx/certs:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./deploy/docker/reverse_proxy/client_max_body_size.conf:/etc/nginx/conf.d/client_max_body_size.conf:ro
acme-companion:
image: nginxproxy/acme-companion
container_name: nginx-proxy-acme
volumes_from:
- nginx-proxy
volumes:
- certs:/etc/nginx/certs:rw
- acme:/etc/acme.sh
- /var/run/docker.sock:/var/run/docker.sock:ro
migrations:
image: mediacms/mediacms:latest
volumes:
- ./:/home/mediacms.io/mediacms/
environment:
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no'
ENABLE_CELERY_BEAT: 'no'
ADMIN_USER: 'admin'
ADMIN_EMAIL: 'admin@localhost'
#ADMIN_PASSWORD: 'uncomment_and_set_password_here'
command: "./deploy/docker/prestart.sh"
restart: on-failure
depends_on:
redis:
condition: service_healthy
db:
condition: service_healthy
web:
image: mediacms/mediacms:latest
deploy:
replicas: 1
volumes:
- ./:/home/mediacms.io/mediacms/
environment:
ENABLE_CELERY_BEAT: 'no'
ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no'
ENABLE_MIGRATIONS: 'no'
VIRTUAL_HOST: 'mediacms.52.209.5.113.nip.io'
LETSENCRYPT_HOST: 'mediacms.52.209.5.113.nip.io'
LETSENCRYPT_EMAIL: 'email@example.com'
depends_on:
- migrations
celery_beat:
image: mediacms/mediacms:latest
volumes:
- ./:/home/mediacms.io/mediacms/
environment:
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_SHORT: 'no'
ENABLE_CELERY_LONG: 'no'
ENABLE_MIGRATIONS: 'no'
depends_on:
- redis
celery_worker:
image: mediacms/mediacms:latest
deploy:
replicas: 1
volumes:
- ./:/home/mediacms.io/mediacms/
environment:
ENABLE_UWSGI: 'no'
ENABLE_NGINX: 'no'
ENABLE_CELERY_BEAT: 'no'
ENABLE_MIGRATIONS: 'no'
depends_on:
- migrations
db:
image: postgres:17.2-alpine
volumes:
- ../postgres_data:/var/lib/postgresql/data/
restart: always
environment:
POSTGRES_USER: mediacms
POSTGRES_PASSWORD: mediacms
POSTGRES_DB: mediacms
TZ: Europe/London
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}", "--host=db", "--dbname=$POSTGRES_DB", "--username=$POSTGRES_USER"]
interval: 30s
timeout: 10s
retries: 5
redis:
image: "redis:alpine"
restart: always
healthcheck:
test: ["CMD", "redis-cli","ping"]
interval: 30s
timeout: 10s
retries: 3
volumes:
conf:
vhost:
html:
dhparam:
certs:
acme:

256
docs/Configuration.md Normal file
View File

@ -0,0 +1,256 @@
## Configuration
A number of options are available on `cms/settings.py`.
It is advisable to override any of them by adding it to `local_settings.py` .
In case of a the single server installation, add to `cms/local_settings.py` .
In case of a docker compose installation, add to `deploy/docker/local_settings.py` . This will automatically overwrite `cms/local_settings.py` .
Any change needs restart of MediaCMS in order to take effect. So edit `cms/local_settings.py`, make a change and restart MediaCMS
```
#systemctl restart mediacms
```
### change portal logo
Set a new svg file for the white theme (`static/images/logo_dark.svg`) or the dark theme (`static/images/logo_light.svg`)
### set global portal title
set `PORTAL_NAME`, eg
```
PORTAL_NAME = 'my awesome portal'
```
### who can add media
By default `CAN_ADD_MEDIA = "all"` means that all registered users can add media. Other valid options are:
- **email_verified**, a user not only has to register an account but also verify the email (by clicking the link sent upon registration). Apparently email configuration need to work, otherise users won't receive emails.
- **advancedUser**, only users that are marked as advanced users can add media. Admins or MediaCMS managers can make users advanced users by editing their profile and selecting advancedUser.
### what is the portal workflow
The `PORTAL_WORKFLOW` variable specifies what happens to newly uploaded media, whether they appear on listings (as the index page, or search)
- **public** is the default option and means that a media can appear on listings. If media type is video, it will appear once at least a task that produces an encoded version of the file has finished succesfully. For other type of files, as image/audio they appear instantly
- **private** means that newly uploaded content is private - only users can see it or MediaCMS editors, managers and admins. Those can also set the status to public or unlisted
- **unlisted** means that items are unlisted. However if a user visits the url of an unlisted media, it will be shown (as opposed to private)
### show/hide the Sign in button
to show button:
```
LOGIN_ALLOWED = True
```
to hide button:
```
LOGIN_ALLOWED = False
```
### show/hide the Register button
to show button:
```
REGISTER_ALLOWED = True
```
to hide button:
```
REGISTER_ALLOWED = False
```
### show/hide the upload media button
To show:
```
UPLOAD_MEDIA_ALLOWED = True
```
To hide:
```
UPLOAD_MEDIA_ALLOWED = False
```
### show/hide the actions buttons (like/dislike/report)
Make changes (True/False) to any of the following:
```
- CAN_LIKE_MEDIA = True # whether the like media appears
- CAN_DISLIKE_MEDIA = True # whether the dislike media appears
- CAN_REPORT_MEDIA = True # whether the report media appears
- CAN_SHARE_MEDIA = True # whether the share media appears
```
### show/hide the download option on a media
Edit `templates/config/installation/features.html` and set
```
download: false
```
### automatically hide media upon being reported
set a low number for variable `REPORTED_TIMES_THRESHOLD`
eg
```
REPORTED_TIMES_THRESHOLD = 2
```
once the limit is reached, media goes to private state and an email is sent to admins
### set a custom message on the media upload page
this message will appear below the media drag and drop form
```
PRE_UPLOAD_MEDIA_MESSAGE = 'custom message'
```
### set email settings
Set correct settings per provider
```
DEFAULT_FROM_EMAIL = 'info@mediacms.io'
EMAIL_HOST_PASSWORD = 'xyz'
EMAIL_HOST_USER = 'info@mediacms.io'
EMAIL_USE_TLS = True
SERVER_EMAIL = DEFAULT_FROM_EMAIL
EMAIL_HOST = 'mediacms.io'
EMAIL_PORT = 587
ADMIN_EMAIL_LIST = ['info@mediacms.io']
```
### disallow user registrations from specific domains
set domains that are not valid for registration via this variable:
```
RESTRICTED_DOMAINS_FOR_USER_REGISTRATION = [
'xxx.com', 'emaildomainwhatever.com']
```
### require a review by MediaCMS editors/managers/admins
set value
```
MEDIA_IS_REVIEWED = False
```
any uploaded media now needs to be reviewed before it can appear to the listings.
MediaCMS editors/managers/admins can visit the media page and edit it, where they can see the option to mark media as reviewed. By default this is set to True, so all media don't require to be reviewed
### specify maximum number of media for a playlist
set a different threshold on variable `MAX_MEDIA_PER_PLAYLIST`
eg
```
MAX_MEDIA_PER_PLAYLIST = 14
```
### specify maximum size of a media that can be uploaded
change `UPLOAD_MAX_SIZE`.
default is 4GB
```
UPLOAD_MAX_SIZE = 800 * 1024 * 1000 * 5
```
### specify maximum size of comments
change `MAX_CHARS_FOR_COMMENT`
default:
```
MAX_CHARS_FOR_COMMENT = 10000
```
### how many files to upload in parallel
set a different threshold for `UPLOAD_MAX_FILES_NUMBER`
default:
```
UPLOAD_MAX_FILES_NUMBER = 100
```
### force users confirm their email upon registrations
default option for email confirmation is optional. Set this to mandatory in order to force users confirm their email before they can login
```
ACCOUNT_EMAIL_VERIFICATION = 'optional'
```
### rate limit account login attempts
after this number is reached
```
ACCOUNT_LOGIN_ATTEMPTS_LIMIT = 20
```
sets a timeout (in seconds)
```
ACCOUNT_LOGIN_ATTEMPTS_TIMEOUT = 5
```
### disallow user registration
set the following variable to False
```
USERS_CAN_SELF_REGISTER = True
```
### configure notifications
Global notifications that are implemented are controlled by the following options:
```
USERS_NOTIFICATIONS = {
'MEDIA_ADDED': True,
}
```
If you want to disable notification for new media, set to False
Admins also receive notifications on different events, set any of the following to False to disable
```
ADMINS_NOTIFICATIONS = {
'NEW_USER': True,
'MEDIA_ADDED': True,
'MEDIA_REPORTED': True,
}
```
- NEW_USER: a new user is added
- MEDIA_ADDED: a media is added
- MEDIA_REPORTED: the report for a media was hit

40
docs/Docker_deployment.md Normal file
View File

@ -0,0 +1,40 @@
# MediaCMS on Docker
The mediacms image is built to use supervisord as the main process, which manages one or more services required to run mediacms. We can toggle which services are run in a given container by setting the environment variables below to `yes` or `no`:
* ENABLE_UWSGI
* ENABLE_NGINX
* ENABLE_CELERY_BEAT
* ENABLE_CELERY_SHORT
* ENABLE_CELERY_LONG
* ENABLE_MIGRATIONS
By default, all these services are enabled, but in order to create a scaleable deployment, some of them can be disabled, splitting the service up into smaller services.
Also see the `Dockerfile` for other environment variables which you may wish to override. Application settings, eg. `FRONTEND_HOST` can also be overridden by updating the `deploy/docker/local_settings.py` file.
See example deployments in the sections below. These example deployments have been tested on `docker-compose version 1.27.4` running on `Docker version 19.03.13`
To run, update the configs above if necessary, build the image by running `docker-compose build`, then run `docker-compose run`
## Simple Deployment, accessed as http://localhost
The main container runs migrations, mediacms_web, celery_beat, celery_workers (celery_short and celery_long services), exposed on port 80 supported by redis and postgres database. The FRONTEND_HOST in `deploy/docker/local_settings.py` is configured as http://localhost, on the docker host machine.
## Advanced Deployment, accessed as http://localhost:8000
Here we can run 1 mediacms_web instance, with the FRONTEND_HOST in `deploy/docker/local_settings.py` configured as http://localhost:8000. This is bootstrapped by a single migrations instance and supported by a single celery_beat instance and 1 or more celery_worker instances. Redis and postgres containers are also used for persistence. Clients can access the service on http://localhost:8000, on the docker host machine. This is similar to [this deployment](../docker-compose.yaml), with a `port` defined in FRONTEND_HOST.
## Advanced Deployment, with reverse proxy, accessed as http://mediacms.io
Here we can use `jwilder/nginx-proxy` to reverse proxy to 1 or more instances of mediacms_web supported by other services as mentioned in the previous deployment. The FRONTEND_HOST in `deploy/docker/local_settings.py` is configured as http://mediacms.io, nginx-proxy has port 80 exposed. Clients can access the service on http://mediacms.io (Assuming DNS or the hosts file is setup correctly to point to the IP of the nginx-proxy instance). This is similar to [this deployment](../docker-compose-http-proxy.yaml).
## Advanced Deployment, with reverse proxy, accessed as https://localhost
The reverse proxy (`jwilder/nginx-proxy`) can be configured to provide SSL termination using self-signed certificates, letsencrypt or CA signed certificates (see: https://hub.docker.com/r/jwilder/nginx-proxy or [LetsEncrypt Example](https://www.singularaspect.com/use-nginx-proxy-and-letsencrypt-companion-to-host-multiple-websites/) ). In this case the FRONTEND_HOST should be set to https://mediacms.io. This is similar to [this deployment](../docker-compose-http-proxy.yaml).
## A Scaleable Deployment Architecture (Docker, Swarm, Kubernetes)
The architecture below generalises all the deployment scenarios above, and provides a conceptual design for other deployments based on kubernetes and docker swarm. It allows for horizontal scaleability through the use of multiple mediacms_web instances and celery_workers. For large deployments, managed postgres, redis and storage may be adopted.
![MediaCMS](images/architecture.png)

20
docs/User_Scenarios.md Normal file
View File

@ -0,0 +1,20 @@
## User scenarios to test
## test video media + image
try uploading a video + image, make sure they get encoded well and check they appear on index/search/category/author page
try editing/setting metadata, confirm action is performed, also that are searchable
try adding custom poster, confirm it loads well on video page/listings
try specifying different thumbnail time, confirm an automatic screenshot is taken
## portal workflow
change workflow to unlisted, check they don't appear on index/search/category/author page
## users management
create an admin, a MediaCMS editor and MediaCMS manager. All should see edit/delete on a media and also comments, and action should work.
For users edit and delete, only MediaCMS manager and admin should see edit/delete and these actions should work.
## test subtitle
add language and test subtitling

File diff suppressed because it is too large Load Diff

View File

@ -1,89 +0,0 @@
# Developer Experience
There is ongoing effort to provide a better developer experience and document it.
## How to develop locally with Docker
First install a recent version of [Docker](https://docs.docker.com/get-docker/), and [Docker Compose](https://docs.docker.com/compose/install/).
Then run `docker compose -f docker-compose-dev.yaml up`
```
user@user:~/mediacms$ docker compose -f docker-compose-dev.yaml up
```
In a few minutes the app will be available at http://localhost . Login via admin/admin
### What does docker-compose-dev.yaml do?
It build the two images used for backend and frontend.
* Backend: `mediacms/mediacms-dev:latest`
* Frontend: `frontend`
and will start all services required for MediaCMS, as Celery/Redis for asynchronous tasks, PostgreSQL database, Django and React
For Django, the changes from the image produced by docker-compose.yaml are these:
* Django runs in debug mode, with `python manage.py runserver`
* uwsgi and nginx are not run
* Django runs in Debug mode, with Debug Toolbar
* Static files (js/css) are loaded from static/ folder
* corsheaders is installed and configured to allow all origins
For React, it will run `npm start` in the frontend folder, which will start the development server.
Check it on http://localhost:8088/
### How to develop in Django
Django starts at http://localhost and is reloading automatically. Making any change to the python code should refresh Django.
If Django breaks due to an error (eg SyntaxError, while editing the code), you might have to restart it
```
docker compose -f docker-compose-dev.yaml restart web
```
### How to develop in React
React is started on http://localhost:8088/ , code is located in frontend/ , so making changes there should have instant effect on the page. Keep in mind that React is loading data from Django, and that it has to be built so that Django can serve it.
### Making changes to the frontend
The way React is added is more complicated than the usual SPA project and this is because React is used as a library loaded by Django Templates, so it is not a standalone project and is not handling routes etc.
The two directories to consider are:
* frontend/src , for the React files
* templates/, for the Django templates.
Django is using a highly intuitive hierarchical templating system (https://docs.djangoproject.com/en/4.2/ref/templates/), where the base template is templates/root.html and all other templates are extending it.
React is called through the Django templates, eg templates/cms/media.html is loading js/media.js
In order to make changes to React code, edit code on frontend/src and check it's effect on http://localhost:8088/ . Once ready, build it and copy it to the Django static folder, so that it is served by Django.
### Development workflow with the frontend
1. Edit frontend/src/ files
2. Check changes on http://localhost:8088/
3. Build frontend with `docker compose -f docker-compose-dev.yaml exec frontend npm run dist`
4. Copy static files to Django static folder with`cp -r frontend/dist/static/* static/`
5. Restart Django - `docker compose -f docker-compose-dev.yaml restart web` so that it uses the new static files
6. Commit the changes
### Helper commands
There is ongoing effort to provide helper commands, check the Makefile for what it supports. Eg
Bash into the web container:
```
user@user:~/mediacms$ make admin-shell
root@ca8c1096726b:/home/mediacms.io/mediacms# ./manage.py shell
```
Build the frontend:
```
user@user:~/mediacms$ make build-frontend
docker compose -f docker-compose-dev.yaml exec frontend npm run dist
> mediacms-frontend@0.9.1 dist /home/mediacms.io/mediacms/frontend
> mediacms-scripts rimraf ./dist && mediacms-scripts build --config=./config/mediacms.config.js --env=dist
...
```

View File

@ -1,155 +0,0 @@
# Developers documentation
## Table of contents
- [1. Welcome](#1-welcome)
- [2. System architecture](#2-system-architecture)
- [3. API documentation](#3-api-documentation)
- [4. How to contribute](#4-how-to-contribute)
- [5. Working with Docker tips](#5-working-with-docker-tips)
- [6. Working with the automated tests](#6-working-with-the-automated-tests)
- [7. How video is transcoded](#7-how-video-is-transcoded)
## 1. Welcome
This page is created for MediaCMS developers and contains related information.
## 2. System architecture
to be written
## 3. API documentation
API is documented using Swagger - checkout ot http://your_installation/swagger - example https://demo.mediacms.io/swagger/
This page allows you to login to perform authenticated actions - it will also use your session if logged in.
An example of working with Python requests library:
```
import requests
auth = ('user' ,'password')
upload_url = "https://domain/api/v1/media"
title = 'x title'
description = 'x description'
media_file = '/tmp/file.mp4'
requests.post(
url=upload_url,
files={'media_file': open(media_file,'rb')},
data={'title': title, 'description': description},
auth=auth
)
```
## 4. How to contribute
Before you send a PR, make sure your code is properly formatted. For that, use `pre-commit install` to install a pre-commit hook and run `pre-commit run --all` and fix everything before you commit. This pre-commit will check for your code lint everytime you commit a code.
Checkout the [Code of conduct page](../CODE_OF_CONDUCT.md) if you want to contribute to this repository
## 5. Working with Docker tips
To perform the Docker installation, follow instructions to install Docker + Docker compose (docs/Docker_Compose.md) and then build/start docker-compose-dev.yaml . This will run the frontend application on port 8088 on top of all other containers (including the Django web application on port 80)
```
docker compose -f docker-compose-dev.yaml build
docker compose -f docker-compose-dev.yaml up
```
An `admin` user is created during the installation process. Its attributes are defined in `docker-compose-dev.yaml`:
```
ADMIN_USER: 'admin'
ADMIN_PASSWORD: 'admin'
ADMIN_EMAIL: 'admin@localhost'
```
### Frontend application changes
Eg change `frontend/src/static/js/pages/HomePage.tsx` , dev application refreshes in a number of seconds (hot reloading) and I see the changes, once I'm happy I can run
```
docker compose -f docker-compose-dev.yaml exec -T frontend npm run dist
```
And then in order for the changes to be visible on the application while served through nginx,
```
cp -r frontend/dist/static/* static/
```
POST calls: cannot be performed through the dev server, you have to make through the normal application (port 80) and then see changes on the dev application on port 8088.
Make sure the urls are set on `frontend/.env` if different than localhost
Media page: need to upload content through the main application (nginx/port 80), and then use an id for page media.html, for example `http://localhost:8088/media.html?m=nc9rotyWP`
There are some issues with CORS too to resolve, in order for some pages to function, eg the manage comments page
```
http://localhost:8088/manage-media.html manage_media
```
### Backend application changes
After I make changes to the django application (eg make a change on `files/forms.py`) in order to see the changes I have to restart the web container
```
docker compose -f docker-compose-dev.yaml restart web
```
## How video is transcoded
Original files get uploaded to the application server, and they get stored there as FileFields.
If files are videos and the duration is greater than a number (defined on settings, I think 4minutes), they are also broken in chunks, so one Encode object per chunk, for all enabled EncodeProfiles.
Then the workers start picking Encode objects and they transcode the chunks, so if a chunk gets transcoded correctly, the original file (the small chunk) gets replaced by the transcoded file, and the Encode object status is marked as 'success'.
original.mp4 (1G, 720px)--> Encode1 (100MB, 240px, chunk=True), Encode2 (100MB, 240px, chunk=True)...EncodeXX (100MB, 720px, chunk=True) ---> when all Encode objects are success, for a resolution, they get concatenated to the original_resolution.mp4 file and this gets stored as Encode object (chunk=False). This is what is available for download.
Apparently the Encode object is used to store Encoded files that are served eventually (chunk=False, status='success'), but also files while they are on their way to get transcoded (chunk=True, status='pending/etc')
(Parenthesis opening)
there is also an experimental small service (not commited to the repo currently) that speaks only through API and a) gets tasks to run, b) returns results. So it makes a request and receives an ffmpeg command, plus a file, it runs the ffmpeg command, and returns the result.I've used this mechanism on a number of installations to migrate existing videos through more servers/cpu and has worked with only one problem, some temporary files needed to be removed from the servers (through a periodic task, not so big problem)
(Parenthesis closing)
When the Encode object is marked as success and chunk=False, and thus is available for download/stream, there is a task that gets started and saves an HLS version of the file (1 mp4-->x number of small .ts chunks). This would be FILES_C
This mechanism allows for workers that have access on the same filesystem (either localhost, or through a shared network filesystem, eg NFS/EFS) to work on the same time and produce results.
## 6. Working with the automated tests
This instructions assume that you're using the docker installation
1. start docker-compose
```
docker compose up
```
2. Install the requirements on `requirements-dev.txt ` on web container (we'll use the web container for this)
```
docker compose exec -T web pip install -r requirements-dev.txt
```
3. Now you can run the existing tests
```
docker compose exec --env TESTING=True -T web pytest
```
The `TESTING=True` is passed for Django to be aware this is a testing environment (so that it runs Celery tasks as functions for example and not as background tasks, since Celery is not started in the case of pytest)
4. You may try a single test, by specifying the path, for example
```
docker compose exec --env TESTING=True -T web pytest tests/test_fixtures.py
```
5. You can also see the coverage
```
docker compose exec --env TESTING=True -T web pytest --cov=. --cov-report=html
```
and of course...you are very welcome to help us increase it ;)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 567 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 509 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 502 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 350 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.3 KiB

Some files were not shown because too many files have changed in this diff Show More