Make the largest loops lazier and async to avoid blocking (#3767)

* lets reduce config calls here Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Lets normalize how we name config attributes across the bot. Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * .... Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Just a tiny PR improving config call in a lot of places (Specially events and Help) Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * stop using `bot.guilds` in `on_command_add` * Just a tiny PR improving config call in a lot of places (Specially events and Help) Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * missed this one Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * nothing to see here Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * lets reduce config calls here Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Just a tiny PR improving config call in a lot of places (Specially events and Help) Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * stop using `bot.guilds` in `on_command_add` * missed this one Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * welp Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * welp Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * welp Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * jack Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Update redbot/cogs/mod/kickban.py Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * Update redbot/cogs/filter/filter.py Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * jack Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * make all large loops async to avoid blocking larger bots Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * ... Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * okay now working AsyncGen Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * may or may not have forgotten black Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Apply suggestions from code review Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * jack's review Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * DOCS Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * DOCS Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Apply suggestions from code review Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * jack Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> * Apply suggestions from code review Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * Update redbot/core/utils/__init__.py Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * Update redbot/core/utils/__init__.py Co-Authored-By: jack1142 <6032823+jack1142@users.noreply.github.com> * avoid loop if possible and if not only iterate once Signed-off-by: Drapersniper <27962761+drapersniper@users.noreply.github.com> Co-authored-by: jack1142 <6032823+jack1142@users.noreply.github.com>
2025-11-21 10:17:59 -05:00 · 2020-04-20 18:56:28 +01:00
parent 465812b673
commit ad979180e5
9 changed files with 175 additions and 21 deletions
--- a/redbot/core/bank.py
+++ b/redbot/core/bank.py
@@ -12,6 +12,7 @@ from . import Config, errors, commands
 from .i18n import Translator

 from .errors import BankPruneError
+from .utils import AsyncIter

 if TYPE_CHECKING:
    from .bot import Red
@@ -405,15 +406,22 @@ async def bank_prune(bot: Red, guild: discord.Guild = None, user_id: int = None)
    global_bank = await is_global()

    if global_bank:
-        _guilds = [g for g in bot.guilds if not g.unavailable and g.large and not g.chunked]
-        _uguilds = [g for g in bot.guilds if g.unavailable]
+        _guilds = set()
+        _uguilds = set()
+        if user_id is None:
+            async for g in AsyncIter(bot.guilds, steps=100):
+                if not g.unavailable and g.large and not g.chunked:
+                    _guilds.add(g)
+                elif g.unavailable:
+                    _uguilds.add(g)
        group = _config._get_base_group(_config.USER)

    else:
        if guild is None:
            raise BankPruneError("'guild' can't be None when pruning a local bank")
-        _guilds = [guild] if not guild.unavailable and guild.large else []
-        _uguilds = [guild] if guild.unavailable else []
+        if user_id is None:
+            _guilds = {guild} if not guild.unavailable and guild.large else set()
+            _uguilds = {guild} if guild.unavailable else set()
        group = _config._get_base_group(_config.MEMBER, str(guild.id))

    if user_id is None: