Tags: officiallymarky/steem-rpc-scanner
Tags
v2.0.0 - Move from Twisted to native AsyncIO + major overhaul All of the project has been migrated from Twisted Reactor to native Python AsyncIO, which affects almost every single file in the project, so I'm not going to go into detail on the individual changes for that. This release is a major overhaul, with many new features, improvements, fixes and more. Not everything is listed below, but most important changes are covered in great detail. **Key Changes** - Many previously hardcoded settings have been changed so that they can be specified via either environment variables, CLI arguments, or both. - Various new environment variables and CLI arguments - Argument parsing in `app.py` and `health.py` have been re-formatted, and now respect the values in `rpcscanner.settings` as defaults. - Node list file - The default node list file has been changed from `nodes.txt` to `nodes.conf` - this allows for IDEs and text editors to apply some syntax highlighting, especially for comments, along with being able to use more assistive IDE features such as keyboard shortcuts to quickly comment / uncomment nodes. - `core.py` now uses regex to extract nodes from the node list, instead of a simple `.readlines()` and a `.strip()` loop. This means in-line comments next to listed RPC nodes are now possible, without causing a problem with the file parsing. - `nodes.txt.example` is now `example.nodes.conf` - which has had many RPC nodes added and removed, plus fancy comment blocks to separate sections of nodes in the file, and inline comments demarcating who runs each node (if known). - `rpcscanner/MethodTests.py` - The instance attribute `MethodTests.METHOD_MAP` has now been refactored into a module level attribute. This allows external RPC method testing functions/methods to be configured, alongside the ones built into the `MethodTests` class. - API Method testing functions now take a first argument `host` - this is to compensate for the newly added capability for adding external RPC method testing functions/methods, by making the pre-existing methods consistent with how an external method would receive the host URL being tested. - Added new `test_all` method, which works similarly to the original `test` method, but tests all supported API methods, or a subset if you specify a whitelist/blacklist. - `rpcscanner/rpc.py` - Classes and functions in this file which previously expected a `reactor` instance to be passed to them, no longer take a `reactor` argument. This is a breaking change, as the order of arguments for various functions/methods/constructors have changed. - General cleanup of `NodePlug._ident_jussi`, including refactoring the server type identification code into a static method which parses a dict/str response. - `health.py` - The default `MAX_SCORE` is now `50` instead of `20` - The scoring algorithm used in `score_node` has been tweaked, and also has a new scoring metric based on whether a node is out of sync, applying a varying score penalty based on how badly out of sync the node is. - New fields `Network` and `PassedStages` have been added to the individual RPC health output - The `Time` field for individual RPC health has been adjusted to show how far behind the node is. - `rpcscanner/RPCScanner.py` - The server scanning stages have been adjusted to provide partial compatibility for older Steem-based networks such as Whaleshares, by using `database_api` instead of `condenser_api`, along with some other small adjustments. - The scanning stages now detect the `network` that a node is on, based on the native currency of the network returning within the dynamic global props. By network, I mean `Steem`, `Hive`, `Whaleshares` etc. - Various new methods such as `add_tasks` and `rpc_tasks` and others, which are helper methods for dealing with AsyncIO, reducing code duplication - `filter_badnodes` and `identify_nodes` have both been cleaned up, and have some new features added to them, related to the new `network` detection. - `plugin_test` now records the timing + retries for individual plugin testing. - App-wide changes - `verbose` and `quiet` have been refactored to behave differently. `verbose` is now more respected, while `quiet` is stricter on the log messages that it allows through - Various changes to the logging system, including the addition of log files, with automatic log folder creation to avoid issues. - Migrated from standard `requirements.txt` to **pipenv**, as pipenv handles both dependency management, and creation/maintenance of virtualenv's. - Added `run.sh` runner script, to make it easier to install, update, and run rpcscanner via pipenv. - Added a `Dockerfile`, so that rpc-scanner can be easily ran within a Docker container on any platform. Pre-built images coming soon. - General reliability and user experience improvements across the application **and various other fixes, changes and improvements...**
VER 1.1 - Improved Ctrl-C handling, better logging, commenting out no… …des support and more. - Overhauled README with feature list, a screenshot, and usage information - Better formatted logging output - Improved categorisation of log messages, so that a reasonable amount of info is printed in non-verbose mode - Added signal handler to improve responsiveness to Ctrl-C - Fixed verbose flag (-v) so you don't have to specify a value after it - Added a flag to specify a custom node list (-f) - Added quiet flag (-q) for minimal logging output - Improved description text in the built-in help (-h)