Feat robots.txt (#237)

* add robots.txt

* removes robots.txt manual steps

cms/urls.py

@@ -2,6 +2,7 @@ import debug_toolbar
 from django.conf.urls import include, url
 from django.contrib import admin
 from django.urls import path, re_path
+from django.views.generic.base import TemplateView
 from drf_yasg import openapi
 from drf_yasg.views import get_schema_view
 from rest_framework.permissions import AllowAny
@@ -15,6 +16,10 @@ schema_view = get_schema_view(
 
 urlpatterns = [
     url(r"^__debug__/", include(debug_toolbar.urls)),
+    path(
+        "robots.txt",
+        TemplateView.as_view(template_name="robots.txt", content_type="text/plain"),
+    ),
     url(r"^", include("files.urls")),
     url(r"^", include("users.urls")),
     url(r"^accounts/", include("allauth.urls")),

docs/Configuration.md

@@ -261,3 +261,10 @@ ADMINS_NOTIFICATIONS = {
 - NEW_USER: a new user is added
 - MEDIA_ADDED: a media is added
 - MEDIA_REPORTED: the report for a media was hit
+
+
+
+### Google Analytics
+
+Checkout the instructions by alberto98fx on [Google Analytics](/docs/robots_and_analytics.md) page.
+

docs/robots_and_analytics.md

@@ -53,68 +53,3 @@
       - ./templates/tracking.html://home/mediacms.io/mediacms/templates/tracking.html
   
  ```
-  
-  
- # Robots.txt
- 
-1. Mediacms is missing robots.txt that can impact your seo. 
-Let's add it: 
-  
-  
-  ``` touch $DIR/mediacms/cms/robots.txt ```
-  
-2. Populate it with what you want: 
-  
-```   
-User-Agent: *
-Allow: /
-Disallow: /admin
-``` 
-  
-3. Inside ```files/views.py``` edit the default import and add HttpResponse under HttpResponseRedirect.
-  
-```
-  from django.http import (
-    HttpResponseRedirect,
-    HttpResponse,
-)
-```
-  
-4. Add a new view:
-  
-```
-  def robots(request):
-    f = open('/home/mediacms.io/mediacms/templates/cms/robots.txt', 'r')
-    file_content = f.read()
-    f.close()
-    return HttpResponse(file_content, content_type="text/plain")
-```
-  
-If you are using Docker the path is the one inside the container. 
-
-5. Open ```files/urls.py``` and add: 
-```
- urlpatterns = [
-    url(r"^robots.txt", views.robots, name="robots"),
-```
-Inside urlpatterns. 
- 
-6. If you are using Docker and did not mount the entire folder before you need to map the volumes:
-  
-```
-  web:
-    image: mediacms/mediacms:latest
-    restart: unless-stopped
-    ports:
-      - "80:80"
-    deploy:
-      replicas: 1
-    volumes:
-      - ./files/views.py:/home/mediacms.io/mediacms/files/views.py
-      - ./files/urls.py:/home/mediacms.io/mediacms/files/urls.py  
-      - ./templates/cms/robots.txt:/home/mediacms.io/mediacms/templates/cms/robots.txt
-```
-  
-7. Call ```https://yourwebsite.com/robots.txt``` to check if everything is working.
-
- 

templates/robots.txt

@@ -0,0 +1,2 @@
+User-Agent: *
+Allow: /