16 Commits

Author SHA1 Message Date
Toby Harradine
d1a46acc9a PostgreSQL driver, tests against DB backends, and general drivers cleanup (#2723)
* PostgreSQL driver and general drivers cleanup

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Make tests pass

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Add black --target-version flag in make.bat

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Rewrite postgres driver

Most of the logic is now in PL/pgSQL.

This completely avoids the use of Python f-strings to format identifiers into queries. Although an SQL-injection attack would have been impossible anyway (only the owner would have ever had the ability to do that), using PostgreSQL's format() is more reliable for unusual identifiers. Performance-wise, I'm not sure whether this is an improvement, but I highly doubt that it's worse.

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Reformat

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Fix PostgresDriver.delete_all_data()

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Clean up PL/pgSQL code

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* More PL/pgSQL cleanup

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* PL/pgSQL function optimisations

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Ensure compatibility with PostgreSQL 10 and below

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* More/better docstrings for PG functions

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Fix typo in docstring

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Return correct value on toggle()

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Use composite type for PG function parameters

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Fix JSON driver's Config.clear_all()

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Correct description for Mongo tox recipe

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Fix linting errors

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Update dep specification after merging bumpdeps

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Add towncrier entries

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Update from merge

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Mention [postgres] extra in install docs

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Support more connection options and use better defaults

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Actually pass PG env vars in tox

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>

* Replace event trigger with manual DELETE queries

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
2019-08-26 22:02:26 -04:00
Michael H
d4b6fdea92 Add towncrier (#2873)
* Adds towncrier as our changelog system.
* Updates our contributor guidelines for this.

Resolves #2872
2019-07-17 11:12:43 +10:00
Toby Harradine
55e309125e
Add uvloop as Posix+CPython dependency and tweak new Make recipes (#2819)
- uvloop is now a dependency on non-Windows CPython systems
- `make setupenv` renamed to `make newenv`
- `make syncenv` added to sync local venv to current dependencies
- `dev-requirements.txt` moved into `tools` directory
2019-07-02 11:53:38 +10:00
Toby Harradine
461f56bca1
Kill Pipfile, update dependencies, and add dep bumping tools (#2806)
### Replacement for pipenv's environment setup
First of all, there's a new Make recipe for all devs and contributors on both Windows and Posix, `make setupenv`, which is kind of a replacement for `pipenv install --dev`. It creates a virtual environment in `.venv` using the inbuilt `venv` module, clearing out any existing virtual environment if needed first. Then it installs all dev dependencies using our new `dev-requirements.txt` file. `CONTRIBUTING.md` has been updated to reflect all of this.

### Dependency version bumping tool
Secondly, I've added a python script, `tools/bumpdeps.py` to help with bumping dependency versions. It has its own Make recipe too, `make bumpdeps`. This script won't work on Windows (yet). It reads the `tools/primary_deps.ini` file, which contains the primary requirements of Red and its extras with loose version specifiers, and outputs all pinned dependencies, in `setup.cfg` format. It's not a foolproof dependency resolver, it's quite simple, but it's bound to help out a lot. It'll try to give warnings if there might be a version conflict, but updating `setup.cfg` with its output and then doing `pip install -r dev-requirements.txt` will allow pip to issue warnings if something is conflicting.

So to add a new dependency, add it to `tools/primary_deps.ini` in the appropriate place, and either use `make bumpdeps` to completely update all dependencies, or simply add it to `setup.cfg` manually with its sub-dependencies, and all versions pinned.

### Sphinx 2.1.2 (docs changes)
The sphinx update brought along the ability to disable type annotations being rendered in function and method signatures, and I have gladly gone and done that. Type annotations should already be specified under the "Parameters" section, and the way sphinx renders them in function signatures makes them much harder to read.

Also, documented classes will now display what classes they inherit from.

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
2019-06-28 00:16:14 +10:00
Toby Harradine
1804314f45
[Audio] Improve Lavalink download/connection exception handling (#2764)
- More errors will be logged to the console with clearer messages when something goes wrong
- Downloading the Lavalink Jar will abort after 5 failed attempts. The connect task will also abort if an unhandled exception occurs whilst downloading or connecting to Lavalink. After this occurs, instead of responding "Connection to Lavalink has not yet been established" to commands, the bot will respond "Connection to Lavalink has failed". This has no effect on other commands which don't involve connecting to Lavalink (e.g. settings commands).
- Logs this message when Lavalink jar is successfully downloaded: `Successfully downloaded Lavalink.jar (<x> bytes written)`
- Uses [`tqdm`](https://github.com/tqdm/tqdm/) to display a progress bar whilst downloading Lavalink.jar.

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
2019-06-23 14:09:59 +10:00
Michael H
16443c8cc0 [CI] Improve automated checks (#2702)
* same stuff, but with some more spurious error supression

* fix issue in permissions found in this

* fix a few more spurious errors

* fix another issue

* semi-spurious error fixes

* .

* formatting

* move this to properly log

* distutils import + virtualenv

* more fixes
2019-06-02 19:42:57 +02:00
aikaterna
65b88c09fb [Audio] Update for Red-Lavalink 0.3.x (#2547)
* [Audio] Update for Red-Lavalink #55

* Update setup.cfg

* Catch all the exceptions

* Update version range on Red-Lavalink

* Catch excepts on Spotify tracks/playlist upload

* No prefix needed
2019-05-08 00:56:40 -04:00
Michael H
ad114295e7
Discord.py dep update 3.1 (#2587)
* Dependency update

discord.py==1.0.1
websockets<7

[style]
black==19.3b0

[Docs]
jinja==2.10.1
urllib3==1.24.2

Changes related to breaking changes from discord.py have also been made
to match

As of this commit, help formatter is back to discord.py's default
2019-04-23 21:40:38 -04:00
Will
6c296a9a17 [V3 Setup] Overhaul backend conversion process through setup scripts (#2579)
* swap to click for setup

* Initial changes

* expose some stuff to allow for per-driver optimizations

* overwrite base config

* add red log

* add one print juuuust in case

* fix this

* thanks kowlin

* damn

* oops

* fix thing

* partial commit

* Working mongo -> json conversion, it sucks tho

* remove unused line

* Wrote initial optimized json importer

* optimized json importer

* remove useless line

* update mongo to json converter

* lets try writing the correct entry

* oops

* style fix

* add some garbage data filters going from old mongo to json

* ignore garbage data in mongov2 conversions

* simplify code a bit and add a completion message

* missed one

* Update pipfile lock

* Lock click version
2019-04-20 20:10:44 -04:00
El Laggron
5a15939f08 Bump aiohttp-json-rpc to version 0.12.1 (#2489) 2019-02-27 09:55:20 +11:00
Caleb Johnson
8e6db0829c [Audio] Connect to lavalink in the background (#2460)
Also:
- restart and reconnect if connection settings change
  - shutdown and restart if not configured to use external
- show a message in [p]play et al. when the connection hasn't been made
- move the JAR download to manager so audio.py can access it
- only start if no process exists
- bump red-lavalink to 0.2.3

Resolves #2306
2019-02-17 09:22:55 +11:00
Michael H
4b831a634a [Audio] Remove players which no longer have a guild. (#2414)
Cleanup players when the bot has one for a guild it leaves.

Bumps Red-Lavalink to v0.2.2
2019-02-16 11:15:33 +11:00
Toby Harradine
9869f95bd6
Update dependencies and copyright year (#2436)
- aiohttp 3.5
- websockets 7
- Rapptz/discord.py@700dbb5
- A few others

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
2019-02-13 10:49:11 +11:00
Toby Harradine
b9d440f2f7
Utilise setup.cfg, move version info to redbot package (#2411)
* Utilise setup.cfg, move version info to redbot package

- `redbot.__init__` now is safe to import without installing dependencies.
- Now deploying binary wheel distribution from travis
- Include locale files in sub-packages of cog packages
- python_requires now has no upper limit

Signed-off-by: Toby Harradine <tobyharradine@gmail.com>
2019-02-09 12:08:22 +11:00
Toby Harradine
7eed033c9e
Prettify README and translate to MarkDown (#2101)
Signed-off-by: Redjumpman <redjumpman@users.noreply.github.com>
2018-09-06 21:53:18 +10:00
Will
d69fd63da7 [V3 Everything] Package bot and write setup scripts (#964)
Ya'll are gonna hate me.

* Initial modifications

* Add initial setup.py

* working setup py help

* Modify setup file to package stuff

* Move a bunch of shit and fix imports

* Fix or skip tests

* Must add init files for find_packages to work

* Move main to scripts folder and rename

* Add shebangs

* Copy over translation files

* WORKING PIP INSTALL

* add dependency information

* Hardcoded version for now, will need to figure out a better way to do this

* OKAY ITS FINALLY FUCKING WORKING

* Add this guy

* Fix stuff

* Change readme to rst

* Remove double sentry opt in

* Oopsie

* Fix this thing

* Aaaand fix test

* Aaaand fix test

* Fix core cog importing and default cog install path

* Adjust readme

* change instance name from optional to required

* Ayyy let's do more dependency injection
2017-09-08 23:14:32 -04:00