[V3] Data Converter (#1293)

* More docstrings

* still not ready...

* push this untested pile of code.

* working, menu needs cleaning up though, modlog converter not here yet

* menu cleanup

* add note about the fact that values are overwritten

* add i18n

* User friendlier quitting

* Better naming of a function

* setup automodule for dataconverter

* More documentation

* use Config.MEMBER (etc) instead of 'MEMBER' (etc)

docs/cog_dataconverter.rst

@@ -0,0 +1,62 @@
+.. Importing data from a V2 install
+
+================================
+Importing data from a V2 install
+================================
+
+----------------
+What you'll need
+----------------
+
+1. A Running V3 bot
+2. The path where your V2 bot is installed
+
+--------------
+Importing data
+--------------
+
+.. important::
+
+    Unless otherwise specified, the V2 data will take priority over V3 data for the same entires
+
+.. important::
+
+    For the purposes of this guide, your prefix will be denoted as 
+    [p]
+
+    You should swap whatever you made your prefix in for this.
+    All of the below are commands to be entered in discord where the bot can
+    see them.
+
+The dataconverter cog is not loaded by default. To start, load it with
+
+.. code-block:: none
+
+    [p]load dataconverter
+
+Next, you'll need to give it the path where your V2 install is.
+
+On linux and OSX, it may look something like:
+
+.. code-block:: none
+
+    /home/username/Red-DiscordBot/
+
+On Windows it will look something like:
+
+.. code-block:: none
+
+    C:\Users\yourusername\Red-DiscordBot
+
+Once you have that path, give it to the bot with the following command
+(make sure to swap your own path in)
+
+.. code-block:: none
+
+    [p]convertdata /home/username/Red-DiscordBot/
+
+
+From here, if the path is correct, you will be prompted with an interactive menu asking you
+what data you would like to import
+
+You can select an entry by number, or quit with any of 'quit', 'exit', 'q', '-1', or 'cancel'

docs/framework_utils.rst

@@ -21,3 +21,9 @@ Mod Helpers
 
 .. automodule:: redbot.core.utils.mod
     :members:
+
+V2 Data Conversion
+==================
+
+.. automodule:: redbot.core.utils.data_converter
+    :members: DataConverter

docs/guide_data_conversion.rst

@@ -0,0 +1,154 @@
+.. Converting Data from a V2 cog
+
+.. role:: python(code)
+    :language: python
+
+============================
+Importing Data From a V2 Cog
+============================
+
+This guide serves as a tutorial on using the DataConverter class
+to import settings from a V2 cog. 
+
+------------------
+Things you'll need
+------------------
+
+1. The path where each file holding related settings in v2 is
+2. A conversion function to take the data and transform it to conform to Config
+
+-----------------------
+Getting your file paths
+-----------------------
+
+You should probably not try to find the files manually.
+Asking the user for the base install path and using a relative path to where the
+data should be, then testing that the file exists there is safer. This is especially
+True if your cog has multiple settings files
+
+Example
+
+.. code-block:: python
+
+    from discord.ext import commands
+    from pathlib import Path
+
+    @commands.command(name="filefinder")
+    async def file_finding_command(self, ctx, filepath):
+        """
+        this finds a file based on a user provided input and a known relative path
+        """
+
+        base_path = Path(filepath)
+        fp = base_path / 'data' / 'mycog' / 'settings.json'
+        if not fp.is_file():
+            pass
+            # fail, prompting user
+        else:
+            pass
+            # do something with the file
+
+---------------
+Converting data
+---------------
+
+Once you've gotten your v2 settings file, you'll want to be able to import it
+There are a couple options available depending on how you would like to convert
+the data.
+
+The first one takes a data path, and a conversion function and does the rest for you.
+This is great for simple data that just needs to quickly be imported without much
+modification.
+
+
+Here's an example of that in use:
+
+.. code-block:: python
+
+    from pathlib import Path
+    from discord.ext import commands
+
+    from redbot.core.utils.data_converter import DataConverter as dc
+    from redbot.core.config import Config
+
+    ...
+            
+    
+        async def import_v2(self, file_path: Path):
+            """
+            to be called from a command limited to owner
+
+            This should be a coroutine as the convert function will
+            need to be awaited
+            """
+
+            # First we give the converter out cog's Config instance.
+            converter = dc(self.config)
+            
+            # next we design a way to get all of the data into Config's internal
+            # format. This should be a generator, but you can also return a single
+            # list with identical results outside of memory usage
+            def conversion_spec(v2data):
+                for guild_id in v2.data.keys():
+                    yield {(Config.GUILD, guild_id): {('blacklisted',): True}}
+                    # This is yielding a dictionary that is designed for config's set_raw. 
+                    # The keys should be a tuple of Config scopes + the needed Identifiers. The
+                    # values should be another dictionary whose keys are tuples representing
+                    # config settings, the value should be the value to set for that.
+
+            # Then we pass the file and the conversion function
+            await converter.convert(file_path, conversion_spec)
+            # From here, our data should be imported
+
+
+You can also choose to convert all of your data and pass it as a single dict
+This can be useful if you want finer control over the dataconversion or want to
+preserve any data from v3 that may share the same entry and set it aside to prompt
+a user
+
+.. code-block:: python
+
+    from pathlib import Path
+    from discord.ext import commands
+
+    from redbot.core.utils.data_converter import DataConverter as dc
+    from redbot.core.config import Config
+
+    ...
+
+    await dc(config_instance).dict_import(some_processed_dict)
+
+
+The format of the items of the dict is the same as in the above example
+
+
+-----------------------------------
+Config Scopes and their Identifiers
+-----------------------------------
+
+This section is provided as a quick reference for the identifiers for default
+scopes available in Config. This does not cover usage of custom scopes, though the
+data converter is compatible with those as well.
+
+Global::
+    :code:`(Config.GLOBAL,)`
+Guild::
+    :code:`(Config.GUILD, guild_id)`
+Channel::
+    :code:`(Config.CHANNEL, channel_id)`
+User::
+    :code:`(Config.USER, user_id)`
+Member::
+    :code:`(Config.MEMBER, guild_id, user_id)`
+Role::
+    :code:`(Config.ROLE, role_id)`
+
+
+-----------------------------
+More information and Examples
+-----------------------------
+
+For a more in depth look at how all of these commands function
+You may want to take a look at how core data is being imported
+
+:code:`redbot/cogs/dataconverter/core_specs.py`

docs/index.rst

@@ -16,6 +16,7 @@ Welcome to Red - Discord Bot's documentation!
     install_debian
     install_centos
     install_raspbian
+    cog_dataconverter
 
 .. toctree::
     :maxdepth: 2
@@ -29,6 +30,7 @@ Welcome to Red - Discord Bot's documentation!
 
     guide_migration
     guide_cog_creation
+    guide_data_conversion
     framework_bank
     framework_bot
     framework_cogmanager