* update windows deps instructions
* changelog
* be more explicit that manual dependency handling is an excersice left to the reader
* let's only grab the MSVC C++ stuff here...
* Meh
* update language
* Update install_windows.rst
* it's really that easy
* create
* add the code
* change de logs
* lets remember linux exists
* fix formtting and change wording
* change the wording a bit more... after running `sphinx-build` it def looks like i used bold a lot
god that was a long commit mesage
* review changes
* draper's review (see description)
i have put or worse partially to scare users and partially as there are other possible effects of a downgrade
* review
* clarity
* feat: dispatch `on_red_api_tokens_update` event on api keys update
* docs: add event reference in Shared API Keys docs
* chore(changelog): add tonwcrier entries
* fix: wrap dispatched api tokens in MappingProxyType
* docs: reflect change of type change to read-only Mapping
* feat(downloader): Install cog from specific commit in repo (initial commit)
- Repo and Installable have commit property now
- New class inheriting from Installable -
InstalledCog (old one from converters.py removed)
- New Repo.checkout() method, which is also async ctx manager
ref #2527
* fix(downloader): Keep information about repo's branch in config
- This is needed to make sure that repo can go back from detached state in some rare unexpected
cases
- current branch is determined by `git symbolic-ref` now as this command errors for detached
HEAD
* feat(downloader): Update repo without cogs, update single cog
The most important part of issue #2527 has been added here
- `[p]repo update` command added
- new conf format - nested dictionary repo_name->cog_name->cog_json
installed libraries are now kept in conf too
- `InstalledCog` renamed to `InstalledModule` - installed libraries use this class
- `Downloader.installed_libraries()` and `Downloader.installed_modules()` added
- `Downloader._add_to_installed()` and `Downloader._remove_from_installed()`
now accept list of modules, of both cogs and libraries
- `[p]cog install` tells about fails of copying cog and installing shared libraries
- `[p]cog update` will truly update only chosen cogs (if provided) or cogs that need update
- pinned cogs aren't checked
- before update, repos are updated
- to determine if update is needed `Repo.get_modified_modules()` is used
- `[p]cog pin` and `[p]cog unpin` commands for pinning/unpinning cogs added
- `Repo.checkout()` allows to choose ctx manager exit's checkout revision
- `Repo.install_cog()` returns `InstalledModule` now and raises CopyingError (maybe breaking?)
- `Repo.install_libraries()` returns 2-tuple of installed and failed libraries (maybe breaking?)
- `RepoManager.get_all_cogs()` added, which returns cogs from all repos
- `RepoManager.repos` property added, which contains tuple of `Repo`
* test(downloader): Repo.current_branch() throws an exception, when branch can't be determined
* style(downloader): rename _add_to_installed to _save_to_installed
This method is used for both adding and updating existing modules in Config
* refactor(downloader): add ctx.typing() for few commands
`[p]cog install` is nested hell, can't wait for moving install logic to separate method
* fix(downloader): refactor and fix `set` usage
* perf(downloader): update commits for ALL checked modules to omit diffs next time
This will also disable running git diff for cogs that have the same commit as the latest one
* style(downloader): few style improvements
- use of mutable object in method definition
- make Repo._get_full_sha1() public method
- too long
line
- don't use len to check if sequence is empty
* feat(downloader): add `[p]cog updateallfromrepos` and `[p]cog updatetoversion` commands
- moved cog update logic into `Downloader._cog_update_logic()` (lack of better name)
- splitted
whole cog update process into smaller methods
- might still need some improvements
- added new
methods to `Repo` class:
- `is_on_branch()` to check if repo is currently checked out to branch
- `is_ancestor()` to check if one commit is ancestor of the other
- fix for
`Downloader._available_updates()` behaviour
broken by commit
5755ab08ba67556b3863e907c6f44d80f4f13d88
* feat(downloader): try to find last commit where module is still present
Enhancements:
- `Installable` now has `repo` attribute containing repo object or `None` if repo is
missing
- `Downloader._install_cogs()` and `Downloader._reinstall_libraries()` are able to install
modules from different commits of repo
- `Repo.checkout()` as ctx manager will now exit to commit
which was active before checking out
- unification of `rev` and `hash` terms:
All function
parameters are explicitly called `hash`, if it can only be commit's full sha1 hash or `rev` if it
can be anything that names a commit object, see
[link](https://git-scm.com/docs/git-rev-parse#_specifying_revisions)
- new
`Repo.get_last_module_occurence()` method, which gets module's Installable from last commit in which
it still occurs
* docs(downloader): Add basic description for `InstalledModule`
* fix(downloader): cog ignored during updates if its commit was missing
After config format update, commit string is empty until update and when such cog was checked and it
wasn't available in repo anymore, it was ignored
* refactor(downloader): Installing cogs from specific rev will pin them
* perf(downloader): Don't checkout when current commit equals target hash
- changes to `Repo.checkout()`:
- `exit_to_rev` is now keyword only argument
- added
`force_checkout` to force checkout even if `Repo.commit` value is the same as target hash
* refactor(downloader): Repo._run() stderr is redirected to debug log now
- added two keyword arguments:
- `valid_exit_codes` which specifies valid exit codes, used to
determine if stderr should be sent as debug or error level in logging
- `debug_only` which
specifies if stderr can be sent only as debug level in logging
* style(downloader): stop using `set` as arg name in `_load_repos()`
* feat(downloader): pass multiple cogs to `[p]cog (un)pin`
* refactor(downloader): accept module name instead of instance, fix spelling
* style(downloader): few small style changes
* fix(downloader): add type annotations + fixes based on them
- fix wrong type annotations and add a lot of new ones
- add checks for `Installable.repo` being `None`
- fix wrong return type in `Downloader._install_requirements`
- show repo names correctly when updating all repos
- fix error when some requirement fails to install
BREAKING CHANGE:
- type of `Repo.available_modules` is now consistent (always `tuple`)
* tests: use same event loop policy as in Red's code
* enhance(downloader): fully handle ambiguous revisions
* build(deps): add pytest-mock dependency to tests extra
* fix(downloader): minor fixes
* feat(downloader): add tool for editing Downloader's test repo
This script aims to help update the human-readable version of repo
used for git integration tests in ``redbot/tests/downloader_testrepo.export``
by exporting/importing it in/from provided directory.
Note
----
Editing `downloader_git_test_repo.export` file manually is strongly discouraged,
especially editing any part of commit directives as that causes a change in the commit's hash.
Another problem devs could encounter when trying to manually edit that file
are editors that will use CRLF instead of LF for new line character(s) and therefore break it.
I also used `.gitattributes` to prevent autocrlf from breaking testrepo.
Also, if Git ever changes currently used SHA-1 to SHA-256 we will have to
update old hashes with new ones. But it's a small drawback,
when we can have human-readable version of repo.
Known limitations
-----------------
``git fast-export`` exports commits without GPG signs so this script disables it in repo's config.
This also means devs shouldn't use ``--gpg-sign`` flag in ``git commit`` within the test repo.
* tests(downloader): add git tests and test repo for them
Also added Markdown file that is even more clear than export file
on what the test repo contains.
This is manually created but can be automated on later date.
* test(downloader): add more tests related to RepoManager
These tests use expected output that is already guaranteed by git tests.
* chore(CODEOWNERS): add jack1142 to Downloader's folders
I know this doesn't actually give any benefit to people that don't have
write permission to the repo but I saw other big fella devs doing this,
so I think this might be advisable.
* enhance(downloader): allow easy schema updates in future
* enhance(downloader): more typing fixes, add comments for clarity
* feat(downloader): add python and bot version check to update process
follow-up on #2605, this commit fully fixes#1866
* chore(changelog): add towncrier entries
* fix(downloader): use `*args` instead of `commands.Greedy`
* fix(downloader): hot-reload issue - `InstallableType` now inherits from `IntEnum`
There's desync of `InstallableType` class types due to hot-reload
and `IntEnum` allows for equality check between different types
* enhance(downloader): ensure there's no cog with same name installed
should fix#2927
* fix(downloader): last few changes before marking as ready for review
* Add MS Azure to the list of hosting providers
* Fix some typos, wording, incorrect commands
* towncrier
* Update docs/getting_started.rst
Co-Authored-By: Michael H <michael@michaelhall.tech>
* Use stable instead of dev version
* changelog entry
* Rename 3020.docs.rst to 3021.docs.rst
* update wording in Getting Started
* add note about package layout
* fix some formatting
* spellcheck
* add cookiecutter note
* Update 3021.docs.rst
* docs: change links to d.py docs to use stable version instead of v1.0.1
* chore(changelog): add towncrier entry
* docs: add |DPY_VERSION| substitution and :dpy_docs: role
* chore(changelog): update towncrier entries to reflect new changes
* Getting started guide
* Remove DigitalOcean referral link
* Fix typos and mispells, thanks to @Flame442
* Remove cogs.red hyperlink until it is finished
* Add towncrier entry
* Add prolog.txt
This is not necessary for this PR but all of the other cog guides rely on this file.
The cog guides are individual branches based on this one, which is why I'm adding this file right now.
* QA changes
* More QA changes
considering -> assuming
red is a girl, not non-binary
* commit
* add launcher depricate to linix/mac
* sorry linux i still ignored you
capitalise some linix stuff i couldnt be bothered to do before
* remove mentions of launcher & update changelog enrty
* Rename `bot.db` as `bot._config`
- Continues work towards strong version guarantees
- Added methods for cog use for a few things which were previously
only accessible via direct access.
- Retained private use in a few internal use locations, though most
methods were updated away from this.
- Updated documentation for shared api token users
* changelog
* more detail
* docstring fixes
* Apparently, I forgot to commit something I had locally
- + a copy/paste failue in the changelog
* *sigh*:
* *sigh*
* [Docs] Update linux install docs, redo venv docs
Some of our pre-req installation docs needed updating on Windows - this adds new sections for Fedora Linux and Debian/Raspbian Buster, and also removes some unnecessary pre-requirements from other distributions. These have all been tested on fresh VPSes, installing Red both in venvs and with --user, and they all seem to work.
Also, apparently the venv docs were too scary before. These changes try to make it clear that it's easier to use than users may think.
This also includes a little note to stop users from accidentally installing Python with pyenv after installing pre-requirements on Ubuntu.
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Add changelog entries
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Update officially supported platforms in README.md
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Combine sections and add openSUSE
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Include example of using `--user` flag
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Use `py -3.7` on Windows outside of venv
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Reorganise changelog entries
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* PostgreSQL driver and general drivers cleanup
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Make tests pass
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Add black --target-version flag in make.bat
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Rewrite postgres driver
Most of the logic is now in PL/pgSQL.
This completely avoids the use of Python f-strings to format identifiers into queries. Although an SQL-injection attack would have been impossible anyway (only the owner would have ever had the ability to do that), using PostgreSQL's format() is more reliable for unusual identifiers. Performance-wise, I'm not sure whether this is an improvement, but I highly doubt that it's worse.
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Reformat
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Fix PostgresDriver.delete_all_data()
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Clean up PL/pgSQL code
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* More PL/pgSQL cleanup
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* PL/pgSQL function optimisations
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Ensure compatibility with PostgreSQL 10 and below
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* More/better docstrings for PG functions
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Fix typo in docstring
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Return correct value on toggle()
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Use composite type for PG function parameters
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Fix JSON driver's Config.clear_all()
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Correct description for Mongo tox recipe
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Fix linting errors
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Update dep specification after merging bumpdeps
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Add towncrier entries
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Update from merge
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Mention [postgres] extra in install docs
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Support more connection options and use better defaults
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Actually pass PG env vars in tox
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Replace event trigger with manual DELETE queries
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* I know this needs a changelog entry and docs still
* update tests for new behavior
* update docs, filter; add changelog
* Ready for review
* stop fetching the same Audit logs when the bot is the mod
* I forgot to press save
* fix a comprehension
* Fix AttributeError
* And the other place that happens
* timing fixes
### Replacement for pipenv's environment setup
First of all, there's a new Make recipe for all devs and contributors on both Windows and Posix, `make setupenv`, which is kind of a replacement for `pipenv install --dev`. It creates a virtual environment in `.venv` using the inbuilt `venv` module, clearing out any existing virtual environment if needed first. Then it installs all dev dependencies using our new `dev-requirements.txt` file. `CONTRIBUTING.md` has been updated to reflect all of this.
### Dependency version bumping tool
Secondly, I've added a python script, `tools/bumpdeps.py` to help with bumping dependency versions. It has its own Make recipe too, `make bumpdeps`. This script won't work on Windows (yet). It reads the `tools/primary_deps.ini` file, which contains the primary requirements of Red and its extras with loose version specifiers, and outputs all pinned dependencies, in `setup.cfg` format. It's not a foolproof dependency resolver, it's quite simple, but it's bound to help out a lot. It'll try to give warnings if there might be a version conflict, but updating `setup.cfg` with its output and then doing `pip install -r dev-requirements.txt` will allow pip to issue warnings if something is conflicting.
So to add a new dependency, add it to `tools/primary_deps.ini` in the appropriate place, and either use `make bumpdeps` to completely update all dependencies, or simply add it to `setup.cfg` manually with its sub-dependencies, and all versions pinned.
### Sphinx 2.1.2 (docs changes)
The sphinx update brought along the ability to disable type annotations being rendered in function and method signatures, and I have gladly gone and done that. Type annotations should already be specified under the "Parameters" section, and the way sphinx renders them in function signatures makes them much harder to read.
Also, documented classes will now display what classes they inherit from.
Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
* Tools for marking things unsafe for general use
* I'm facepalming so much...
Actually, make the two do something different
instead of getting distracted writing different docs for both based on intended usage.
* local scopes mmkay + tests
* Move file to adress feedback
* typo fix
* Update __init__.py
* Fix issue with exported names in __init__
* changelog
* [Bot] Support new design
* [Context] use the new help in `ctx.send_help`
* [Commands] Update Cog and Group for help compat
- Removes a trap with all_commands, this isn't a good way to check this
- Adds a help property
- Fixes command parsing in invoke
* Redesigns red's help
* handle fuzzy help
* style
* handle a specific ugly hidden interaction
* fix bot-wide help grouping
* changelog
* remove no longer needed -
* Add release notes for major functionality changes and instructions
* Update docs/changelog_3_1_0.rst
Co-Authored-By: Michael H <michael@michaelhall.tech>
* feat(downloader): add `min_bot_version` and `max_bot_version`
Adds actually working way of specifying minimum and maximum bot version and removes not working
`bot_version`
BREAKING CHANGE: - removal of `bot_version` attribute in `Installable`
* test(downloader): `Installable` tests fix for new bot version attributes
* docs(changelog): added changelog entries for this PR
* [Streams] Add ability to exclude rerun streams from alerts
* [Docs] Changelog entries for contributions by EgonSpengler
* Changelog entry for #2620 [Streams] Ignore Reruns In Alerts
* docs(changelog): [Mod] Allow admin to choose amount of repeats for "deleterepeats" #2437
* docs(changelog): Spelling correction of method name in Tunnel #2496
* docs(changelog): Tunnel fix - When tunnel closes, message should be sent to other end #2507
* docs(changelog): [V3 Downloader] Tell user how to load the cog after [p]cog install #2523
* docs(changelog): [V3 Audio] If bot has move members perm, it can join to user-limited channels #2525
* docs(changelog): [Trivia] Fix of dead image link (world flags) #2540
* docs(changelog): [V3 Test] Make sure that trivia test will use utf-8 encoding #2565
* docs(changelog): [V3 Core] Print actual version, when `--version` flag is used #2567
* docs(changelog): [V3 Downloader] Stop including subpackages in cog list #2590
* docs(changelog): [V3 Downloader] Uninstall multiple cogs #2592
* docs(changelog): [V3 Downloader] Always remove cog from installed in `[p]cog uninstall` #2595
