Logging enhancements and cleanup (#2502)

* Logging enhancements and cleanup

- Removed debug log messages every time `Config.get_conf` is used or a JSON file is read/saved. The basic configuration is now logged once with DEBUG when the bot starts up instead.
- Changed logging output format to reverse date order, include seconds, and use the logger's name instead of the module, function and line number.
- Log files are now kept in the `DATAPATH/core/logs` directory. Each time Red is restarted, a new log is created, and the old ones renamed in a rotating fashion. There can be a maximum of 9 logs in total.
- Each log file now has a smaller max size of 500KB before it will be split into multiple parts. There are also a maximum of 9 parts of each log.
- Discord.py logger now uses the same output formatter as red's loggers
- Moved logging setup code into `redbot.logging` module.

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Reformat

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Implement discussed changes

- We maintain a red.log over multiple runtimes, alongside a latest.log and previous.log for individual runtimes.
- Naming convention changed a bit. E.g. when latest.log is just one part, it will be named latest.log. When it becomes two parts, they will both be named latest-part1.log and latest-part2.log.
- Rotation direction is reversed. This means as the files end up being named in chronological order.

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

redbot/core/cli.py

@@ -1,5 +1,6 @@
 import argparse
 import asyncio
+import logging
 
 
 def confirm(m=""):
@@ -97,7 +98,14 @@ def parse_cli_flags(args):
         "login. This is useful for testing the boot "
         "process.",
     )
-    parser.add_argument("--debug", action="store_true", help="Sets the loggers level as debug")
+    parser.add_argument(
+        "--debug",
+        action="store_const",
+        dest="logging_level",
+        const=logging.DEBUG,
+        default=logging.INFO,
+        help="Sets the loggers level as debug",
+    )
     parser.add_argument("--dev", action="store_true", help="Enables developer mode")
     parser.add_argument(
         "--mentionable",

redbot/core/config.py

@@ -569,13 +569,9 @@ class Config:
         # We have to import this here otherwise we have a circular dependency
         from .data_manager import basic_config
 
-        log.debug("Basic config: \n\n{}".format(basic_config))
-
         driver_name = basic_config.get("STORAGE_TYPE", "JSON")
         driver_details = basic_config.get("STORAGE_DETAILS", {})
 
-        log.debug("Using driver: '{}'".format(driver_name))
-
         driver = get_driver(
             driver_name, cog_name, uuid, data_path_override=cog_path_override, **driver_details
         )

redbot/core/json_io.py

@@ -47,7 +47,6 @@ class JsonIO:
         And:
             https://www.mjmwired.net/kernel/Documentation/filesystems/ext4.txt#310
         """
-        log.debug("Saving file {}".format(self.path))
         filename = self.path.stem
         tmp_file = "{}-{}.tmp".format(filename, uuid4().fields[0])
         tmp_path = self.path.parent / tmp_file
@@ -80,7 +79,6 @@ class JsonIO:
 
     # noinspection PyUnresolvedReferences
     def _load_json(self):
-        log.debug("Reading file {}".format(self.path))
         with self.path.open(encoding="utf-8", mode="r") as f:
             data = json.load(f)
         return data