Skip to content

Tags: officiallymarky/steem-rpc-scanner

Tags

v2.0.0

v2.0.0 - Move from Twisted to native AsyncIO + major overhaul

All of the project has been migrated from Twisted Reactor to native Python AsyncIO, which affects
almost every single file in the project, so I'm not going to go into detail on the individual changes
for that.

This release is a major overhaul, with many new features, improvements, fixes and more. Not everything
is listed below, but most important changes are covered in great detail.

**Key Changes**

 - Many previously hardcoded settings have been changed so that they can be specified via either
   environment variables, CLI arguments, or both.
 - Various new environment variables and CLI arguments
 - Argument parsing in `app.py` and `health.py` have been re-formatted, and now respect
   the values in `rpcscanner.settings` as defaults.

 - Node list file
    - The default node list file has been changed from `nodes.txt` to `nodes.conf` - this
        allows for IDEs and text editors to apply some syntax highlighting, especially for comments,
        along with being able to use more assistive IDE features such as keyboard shortcuts to quickly
        comment / uncomment nodes.
    - `core.py` now uses regex to extract nodes from the node list, instead of a simple `.readlines()`
      and a `.strip()` loop. This means in-line comments next to listed RPC nodes are now possible,
      without causing a problem with the file parsing.
    - `nodes.txt.example` is now `example.nodes.conf` - which has had many RPC nodes added and removed,
      plus fancy comment blocks to separate sections of nodes in the file, and inline comments
      demarcating who runs each node (if known).

 - `rpcscanner/MethodTests.py`
    - The instance attribute `MethodTests.METHOD_MAP` has now been refactored into a module level
      attribute. This allows external RPC method testing functions/methods to be configured, alongside
      the ones built into the `MethodTests` class.
    - API Method testing functions now take a first argument `host` - this is to compensate for
      the newly added capability for adding external RPC method testing functions/methods, by
      making the pre-existing methods consistent with how an external method would receive the
      host URL being tested.
    - Added new `test_all` method, which works similarly to the original `test` method, but
      tests all supported API methods, or a subset if you specify a whitelist/blacklist.

 - `rpcscanner/rpc.py`
    - Classes and functions in this file which previously expected a `reactor` instance to be passed
      to them, no longer take a `reactor` argument. This is a breaking change, as the order of
      arguments for various functions/methods/constructors have changed.
    - General cleanup of `NodePlug._ident_jussi`, including refactoring the server type identification
      code into a static method which parses a dict/str response.

 - `health.py`
    - The default `MAX_SCORE` is now `50` instead of `20`
    - The scoring algorithm used in `score_node` has been tweaked, and also has a new scoring metric based
      on whether a node is out of sync, applying a varying score penalty based on how badly out of sync
      the node is.
    - New fields `Network` and `PassedStages` have been added to the individual RPC health output
    - The `Time` field for individual RPC health has been adjusted to show how far behind the node is.

 - `rpcscanner/RPCScanner.py`
    - The server scanning stages have been adjusted to provide partial compatibility for older Steem-based networks
      such as Whaleshares, by using `database_api` instead of `condenser_api`, along with some other small adjustments.
    - The scanning stages now detect the `network` that a node is on, based on the native currency of the network
      returning within the dynamic global props. By network, I mean `Steem`, `Hive`, `Whaleshares` etc.
    - Various new methods such as `add_tasks` and `rpc_tasks` and others, which are helper methods for dealing with
      AsyncIO, reducing code duplication
    - `filter_badnodes` and `identify_nodes` have both been cleaned up, and have some new features added to them, related
      to the new `network` detection.
    - `plugin_test` now records the timing + retries for individual plugin testing.

 - App-wide changes
    - `verbose` and `quiet` have been refactored to behave differently. `verbose` is now more respected,
      while `quiet` is stricter on the log messages that it allows through
    - Various changes to the logging system, including the addition of log files, with automatic
      log folder creation to avoid issues.
    - Migrated from standard `requirements.txt` to **pipenv**, as pipenv handles both dependency management,
      and creation/maintenance of virtualenv's.
    - Added `run.sh` runner script, to make it easier to install, update, and run rpcscanner via pipenv.
    - Added a `Dockerfile`, so that rpc-scanner can be easily ran within a Docker container on any platform.
      Pre-built images coming soon.
    - General reliability and user experience improvements across the application

**and various other fixes, changes and improvements...**

v1.5.1

v1.5.1 - some speed improvements, and less spammy plugin logs in non-…

…verbose mode

v1.5

v1.5 - new health.py tool, for generating easily parsed lists of work…

…ing nodes, or scanning individual nodes with unix return codes

v1.4

Version 1.4 - Major refactoring, plus new --plugins switch, for thoro…

…ugh node scanning

v1.1

VER 1.1 - Improved Ctrl-C handling, better logging, commenting out no…

…des support and more.

 - Overhauled README with feature list, a screenshot, and usage information
 - Better formatted logging output
 - Improved categorisation of log messages, so that a reasonable amount of info is printed
   in non-verbose mode
 - Added signal handler to improve responsiveness to Ctrl-C
 - Fixed verbose flag (-v) so you don't have to specify a value after it
 - Added a flag to specify a custom node list (-f)
 - Added quiet flag (-q) for minimal logging output
 - Improved description text in the built-in help (-h)